Regex matching behavior with multi-character unicode symbol

user_3134786 0
Viewed 96

I am having trouble understanding some behavior I observed with multi-character unicode symbols.

Take, as an example, the string , and the regex (|)(?![]), I get three matches: Both flags, and the last hand. Expected: 5 matches, each symbol once.

Since both and are 2 character symbols, I tried writing a non-unicode example. With the string abcdabcdab and the regex (ab|cd)(?![b]), I get the expected 5 matches, each pair of ab and cd once.

Thinking that there might be some interaction between and , I used a different unicode character, giving me the regex (|)(?![]). Here I get the same result that I got in the first example.

Since both and are usually not used individually, I tried using "normal" unicode or ASCII characters instead of . I my example, I used and a, which gave me the expected result of 5 matches, each symbol once.

Is someone able to explain this behavior, or is this a bug?

This behavior only happened in PCRE and the JavaScript regex engine, I used this site to test it. https://regex101.com/

pcre unicode regex
1 Answers

You should not put a multibyte character inside a character class like in (?![]). Inside the character class, it got "decomposed" into a sequence of two bytes, \uD83C and \uDFFE , matching either of them, not as a sequence. As the hand emoji is a sequence of \uD83E\uDD1A\uD83C\uDFFE (it ends with these two bytes), the lookahead got triggered and affected the matches.

To solve the problem, you just need to remove the brackets and use (|)(?!) so that the char could be treated as a byte sequence, not one or another char.

user_38329700
Related