Perl: Fastest match of anything?

user_363028 0
Viewed 87

I want to do this without using /s:

(?:.|\n)+

And I want it to be fast.

It is part of a bigger regexp, which is the reason I cannot use /s. I have tested:

perl -pe 's/(?:.|\n)+//'  #  30 MBytes/s
perl -pe 's/[^\777]+//'   # 124 MBytes/s

They are not as fast as /s:

perl -pe 's/.+//s'        # 164 MBytes/s

Can I somehow get the same speed as /s?

Edit:

perl -pe 's/(?s).+(?-s)//' # 162 MBytes/s
perl -pe 's/[\d\D]+//'    # 162 MBytes/s
perl -pe 's/[\s\S]+//'    # 161 MBytes/s

These are all good options. Thanks.

perl regex performance
3 Answers

You do not need to use anyworkarounds.

You may use inline modifier flags, (?s) to enable dot to match across lines and (?-s) to disable this effect.

For example:

(?s).*PATTERN(?-s).*

where the first .* matches any text and the last .* only matches till the end of a line.

You can also use modifier groups:

(?s:.*)PATTERN(?-s:.*)

See more in Extended Patterns.

user_38329700

You can use (?s:PATTERN) to enable /s for just the subpattern in the parens.

(?s:.)+

or

(?s:.+)

For example,

$ perl -M5.010 -e'
   say /^.(?s:.).\z/ ? "match" : "no match"
      for "\n\n\n", "A\n\n", "\n\nA", "A\nA";
'
no match
no match
no match
match

There is no difference between . with /s and (?s:.), so you'll get exactly the same performance.

$ perl -Mre=debug -e'qr/(?s:.)/'
Compiling REx "(?s:.)"
Final program:
   1: SANY (2)
   2: END (0)
minlen 1
Freeing REx: "(?s:.)"

$ perl -Mre=debug -e'qr/./s'
Compiling REx "."
Final program:
   1: SANY (2)
   2: END (0)
minlen 1
Freeing REx: "."

Advantages over the other solutions:

  • (?s:.) restores the s flag state, while (?s).(?-s) turns it off.
  • (?s:.) is a fair bit shorter than (?s).(?-s).
  • (?s:.) is not longer than [\s\S] and [\d\D].
  • (?s:.) is not a workaround like [\s\S] and [\d\D].

That said, (?s:.), (?s).(?-s), [\s\S] and [\d\D] all result in exactly the same regex program (since 5.24), so they'll all perform identically.

user_5899240

If I create a 98MB file:

$ dd bs=1024 count=100000 </dev/urandom >file
100000+0 records in
100000+0 records out
102400000 bytes transferred in 0.558640 secs (183302305 bytes/sec)

Now compare some methods:

time perl -0777 -lne 's/(?:.|\n)+//' file 
real    0m1.957s
user    0m1.902s
sys 0m0.050s

time perl -0777 -lne 's/[^\777]+//' file 
real    0m0.130s
user    0m0.082s
sys 0m0.046s

time perl -0777 -lne 's/[\s\S]+//' file 
real    0m0.065s
user    0m0.022s
sys 0m0.041s

time perl -0777 -lne 's/.+//s'  file 
real    0m0.064s
user    0m0.022s
sys 0m0.038s

As you can see, the character class of [\s\S] is optimized to /./s since 5.24.

$ perl -Mre=debug -e'qr/./s'
Compiling REx "."
Final program:
   1: SANY (2)
   2: END (0)
minlen 1
Freeing REx: "."

$ perl -Mre=debug -e'qr/[\s\S]/'
Compiling REx "[\s\S]"
Final program:
   1: SANY (2)
   2: END (0)
minlen 1
Freeing REx: "[\s\S]"
user_2986070
Related