RegEx expression to remove duplicate word groups

user_16180386 0
Viewed 263

I am trying to remove duplicate values from the following text:

[John Smith][John Smith][Jane Doe][Jane Doe][Doe][John][Smith John][John Smith Sr]

However, I need to treat each group of words inside the brackets as a unique value. That way, if a single word appears inside multiple brackets, but the group of words inside the brackets differ, I keep all values.

Using my sample text I expect to get the below,

[John Smith][Jane Doe][Doe][John][Smith John][John Smith Sr]

But instead I get the following text when using this RegEx (?:\[)(.+)(?:\])(?=.*\b\1,?)

[Jane Doe][Doe][Smith John][John Smith Sr]

Where am I going wrong? Appreciate any help I can get.

regex
2 Answers

You have to capture also the squared brakets in the first group:

(\[.+\])(?=.*\1)

In this way you ensure to check the duplicates w.r.t. the whole brakets' content.

user_46077330

Capture each encountered square-braced substring, then check if that same substring occurs again later in the string -- if so, replace it.

(\[[^]]+])(?=.*\1)

Using a negated character class ([^]]) allows the use of a greedy quantifier + without "over-matching" -- this means that the regex won't waste time trying to match multiple consecutive square braces later in the string before backtracking to smaller potential matches.

https://regex101.com/r/OD1hia/1

If using javascript or another environment that requires escaping the closing brace in the negated character class, add a slash there.

https://regex101.com/r/Wm3K4f/1

There will be many duplicates of this technique on Stack Overflow, but I didn't find an exact "snowflake" duplicate that requires the square braces.

user_29434030
Related