Bug in Notepad++ / BOOST or bug in my regular expression?

Question

I have a file which is structured like this:

Line
foo Änderbar: PM baz
Line

Line
foo Änderbar: OM baz
Line

Line
foo Änderbar: ++ baz
Line

Line
foo Änderbar: -- baz
Line

So the file consists of "blocks" which are separated by a newline (I have converted the file to Unix line endings). Each block can have an arbitrary number of lines. Each line of a block contains at least one character which is not a newline, and is finished by a newline character. The lines which separate the blocks consist of exactly one newline character.

In each block, there is exactly one line in the following format:

at least one character which is not newline, followed by
the literal string 'Änderbar: ', followed by
exactly one of the literal strings '++', '--', 'OM', 'PM', followed by
at least one character which is not newline, followed by
the line-terminating newline character

There is always at least one other non-empty line in the same block above this special line and one other non-empty line below this special line.

I need an effective method to find (and thereby select) all blocks where the literal after Änderbar: is -- (find / select one block after another, each one after hitting Find Next again, i.e. not selecting all of those blocks at the same time).

Normally, I have fun solving such problems with Notepad++. However, in that case, it seems that I either get more and more stupid as I get older, or that there is a bug in Notepad++'s regex handling engine.

Notepad++ uses BOOST (and supports PCRE expressions via BOOST). Since this is in wide use, I consider that problem important enough to post it here, just in case that BOOST really is the reason for the misbehavior.

Having said this: I loaded that file into Notepad++, fired up the Search and Replace dialog, ticked . matches newline, ticked Regular Expression and entered the following regex in the Find What: textbox:


([^
]+
)+[^
]+(Änderbar\:\ --[^
]+
)([^
]+
)+

I was quite surprised that this made Notepad++ behave weirdly: When the cursor was placed in the empty line immediately before a block with Änderbar: --, hitting Find Next found / selected that block as expected. But when the cursor was at another place, hitting Find Next made Notepad++ find / select the whole rest of the file, i.e. all blocks below the cursor position.

I then have tested if it would find the blocks having ++ after Änderbar:, i.e. I changed my regex to


([^
]+
)+[^
]+(Änderbar\:\ \+\+[^
]+
)([^
]+
)+

Guess what: This was working reliably in each situation. The same is true for the last both:


([^
]+
)+[^
]+(Änderbar\:\ PM[^
]+
)([^
]+
)+

([^
]+
)+[^
]+(Änderbar\:\ OM[^
]+
)([^
]+
)+

So Notepad++ / PCRE seems to have a problem with the correct interpretation of - under certain circumstances, or I have a subtle bug in my regex which only triggers when I am searching for -- (instead of ++, OM or PM) at the respective place.

Please note that I already have tried to leave away the \ in front of the space character (which actually could only make the situation worse, but I've tried just in case) and that I also have tried to use \-\- instead of -- (although the latter should be fine). That did not alter the (mis-)behavior in any way.

So what is the problem here? Is there a bug in my regex, or is there a bug in Notepad++?

UPDATE

I have stripped down the actual file in question and have uploaded it to https://pastebin.com/w62E57U5. To reproduce the problem, please do the following:

Download the file from the link above and save it somewhere on your HDD (do not copy the text directly into Notepad++).
Load the file into Notepad++. The cursor now is in the topmost line, and nothing is selected.
This is essential: Click Edit -> EOL Conversion -> Unix (LF).
Verify that the cursor is still in the topmost line (which is empty) and that nothing is selected.
Open the Find dialog and choose the settings and enter the search string as described above.
Click "Find Next".
Note that now the complete text is found / selected.
Keeping the Find window open, delete the third line of the file (it reads "Funktionspaket(e): ML"). Do not just empty that line, but really delete it so that no empty line remains between the line before and the line after.
Again, place the cursor in the topmost line (which is still empty) and make sure nothing is selected.
Click "Find Next".
Note that the regular expression now works as expected.

Obviously, somebody is trying to make a fool of me, right?

user_1426469 · Answer

It's not a bug. You're just forgetting something very important - with Windows line endings, your lines have a before the , so the ([^ ]+ )+ part of your RegEx will also match your blank lines which is why clicking "Find Next" matches everything from the cursor position instead of from the start of the block.

Go to Edit > EOL Conversion > Unix (LF) and you'll see that it works now. If you want to support Windows and Unix line endings you'll have to change every [^ ] to [^ ] and every to ?.

user_372239 · Answer

I think the key is: you need to begin your regex with ^ (beginning of line).

Your original regex becomes:

^
([^
]+
)+[^
]+(Änderbar\:\ --[^
]+
)([^
]+
)+

But you can simplify it with:

^\R(?:.+\R)+.+Änderbar: --.+\R(?:.+(?:\R|\z))+

Note: tick `. matches newline`

Where:

\R matches any kind of linebreak, no needs to change the EOL.
\z matches the end of file, if you don't use it, you can't match the last line of the file if there're no linebreak.
(?:...) is a non capture group, much more efficient (if you don't need to capture, of course)

Both works fine with your 2 sample files.

Bug in Notepad++ / BOOST or bug in my regular expression?

2 Answers

Note: tick . matches newline

Note: tick `. matches newline`