Why are these two regexes not the same?

user_2191572 0
Viewed 94

I recently answered this question and it got me thinking about emulating variable-width lookbehinds.

Doesn't work:

(?:(?<=<\/)|(?<=<))[^ >]+

https://regex101.com/r/1wQz5E/1/

Works:

(?:(?<=<\/)|(?<=<))[^ >\/]+

https://regex101.com/r/1wQz5E/2

The expectation is for the closing tag's forward slash to be omitted from the match.

I am trying to wrap my head around why my first example fails to behave as desired.

Source string:

<!DOCTYPE html>

<html lang="en" xmlns="http://www.w3.org/1999/xhtml">
<head>
    <meta charset="utf-8" />
    <title></title>
</head>
<body>
    <a></a>
</body>
</html>

pcre regex
2 Answers

First of all, your first regex can be shortened to (?<=<\/|<)[^ >]+, it is easier to work with shorter patterns.

Keep in mind, the regex engine parses the string from left to right. The (?<=<\/|<) lookbehind matches a location that is immediately preceded with </ or <, i.e. a location right after < or </.

When the input is </a>, the position after < is found with the (?<=<\/|<) pattern first (as it is more to the left than the position after </), and then [^ >]+ consumes / and all what it can match, and [^ >]+ can match / and <, all the chars that the lookbehind pattern matches.

What matches first in an alternation wins, this rule is not broken here, just the alternation happens to be in a non-consuming pattern here, and that bit should also be kept in mind.

A rule of thumb here is: alternation, or unknown length quantifiers like +/*/{1,x} at the end of the lookbehind patterns that are immediately followed with a consuming pattern that also matches those trailing lookbehind patterns, make the lookbehind patterns match the fewest chars, and the subsequent consuming patterns tend to start grabbing content as early as they can.

In such cases, the safest solution is to use consuming patterns instead of infinite length lookbehind together with a capturing group to grab the requied part of the match:

(?:<\/|<)([^ >]+)
<\/?([^ >]+)

Or, in PCRE/Onigmo/PyPi Python regex, you may also use \K:

<\/?\K[^ >]+

Here, the non-capturing (?:<\/|<) group or <\/? pattern will match and consume < or </, and the ([^ >]+) capturing group will yield the expected substring since it won't have a chance to grab the text right after <. \K just omits all text matched so far from the overall match memory buffer, so the only text returned is the one matched with [^ >]+.

However, in this case, you can also solve the problem with the lookbehinds by requiring the first char matched with [^ >]+ to be a char other than /:

(?<=<|</)[^ >/][^ >]*

So, basically unrolling [^ >/]+ with [^ >/][^ >]*.

user_38329700

For the current examples, in this pattern (?:(?<=<\/)|(?<=<))[^ >]+ looking back from the current position, what is directly to the left can be only either </ or < so only 1 of the assertions can be true at the same time.

When it encounters < in </title> the second assertion is true. When trying to match it can not, due to [^ >\/] that does not allow /, then it moves to the next position because there is no more of the pattern to process.

Next thing is </ which is true for the first assertion, and [^ >\/]+ can match the following chars.

So the reason it does not yield the expected matches is because in this pattern (?:(?<=<\/)|(?<=<))[^ >]+ the lookbehind is the same, but the negated character class [^ >]+ can match the / in </title>

user_54249880
Related