Workaround for negative lookbehind limitation in python regexes

user_1052117 0
Viewed 114

I am writing a regex to identify problems in a text, but only if there is no negation within n=3 words before my phrase of interest. Here's what I have so far:

regex = r'''(?ix)     # case insensitive, verbose mode
\s+?
(?<!(not|no|never)){1,3}  # if this is within 3 words, you do not match, negative lookbehind
\s+?
(a|the|any|my|your)  # articles
\s+?
(issue|issues|problem|problems) # words of interest
'''

Should match:

matches = [
"a problem",
"the issue",
"any of the issues",
"not even close to being your issue",
]

Should not match:

non_matches = [
  "not a problem",
  "never your problem",
  "not the issue",
  "not overwhelmingly your issue",
  "not too close your issue"
]

If I run without the negative look-behind:

regex2 = r'''(?ix)  # case insensitive, verbose
(a|the|any|my|your)    # articles
\s+?
(issue|issues|problem|problems) # words of interest
'''

I get the correct positive matches.

>>> for p in matches:
...   print(re.findall(regex2, p))
[('a', 'problem')]
[('the', 'issue')]
[('the', 'issue')]
[('your', 'issue')]

But if I include the negative lookahead I need in order to correctly exclude the negative matches, I get:

re.error: look-behind requires fixed-width pattern

I understand this is just a limitation of the python regex engine, but what is the appropriate workaround commonly used in this situation? Is there a simple way I can OR together 0,1,2,3 patterns to handle it? Something else?

python regex
2 Answers

You may use this work-around approach in Python in the absence of a dynamic length lookbehind support:

regex = r'''(?ixm)
^
(?!.*
   \b(?:not?|never)\s+
   (?:\w+\s+){0,2}
   (?:a|the|any|my|your)\s+
   (?:issues?|problems?)
)
.*\b(a|the|any|my|your)
\s+
(issues?|problems?)
'''

RegEx Demo

Here we are using a negative lookahead at the start of the regex to fail the match if we have non-allowed pattern present in the input.

(?!.*
   \b(?:not?|never)\s+
   (?:\w+\s+){0,2}
   (?:a|the|any|my|your)\s+
   (?:issues?|problems?)
)

This will fail a match when we have no or not or never within 1 to 3 words of an article followed by word of your interest.

user_5482250

You can install PyPi regex module and forget about the lookbehind pattern limitations.

import regex
rx = r'''(?ix)     # case insensitive, verbose mode
(?<!\b(?:not|no|never)(?:\s+\S+){0,2}\s+)  # if this is within 3 words, you do not match, negative lookbehind
(a|the|any|my|your)  # articles
\s+
(issue|issues|problem|problems)\b # words of interest
'''

and then

for p in matches:
    print(regex.findall(rx, p))

# [('a', 'problem')]
# [('the', 'issue')]
# [('the', 'issues')]
# [('your', 'issue')]

for p in non_matches:
    print(regex.findall(rx, p))

# []
# []
# []
# []
# []

See the Python demo.

NOTES:

  • (?<!\b(?:not|no|never)(?:\s+\S+){0,2}\s+) is a negative lookbehind that fails the match if the current location is immediately preceded with a whole word not, no or never and then has zero, one or two repetitions of one or more whitespaces followed with one or more non-whitespaces and then one or more whitespaces. So, all in all the negative words can appear up to 3 words far from the words of interest.
  • Use non-capturing groups, (?:...), around the pattern parts you do not need to extract, as re.findall and regex.findall always return only captures if capturing groups are defined in the pattern. Else, you would need a .finditer method.
  • Pay attention to (issue|issues|problem|problems)\b: the word boundary at the end makes the regex engine match these words as whole words, and if thereis issues, it will return this word in full. If you do not use \b and you still want to get [('the', 'issues')] as the result of the third matches string you will need to put issues alternative before the issue one as the first alternative matched "wins" and the rest is not tried.
user_38329700
Related