Problem with the outputs from using double square brackets

Question

I was told that [[a-z][0-9]] is equivalent to [a-z0-9], but I tried a few examples:

grepl("[[a-z][0-9]]", "d") returns FALSE.

Similarly, grepl("[[:alpha:][0-9]]", "d") returns FALSE while things like grepl("[[:upper:][:lower:]]", "d") works fine.

May I ask if this indicates that double square brackets could only be used for combining things of the form "[:...:]" but not for things like [A-z] or [0-9]?

If so, why would R stop us from doing so? And what do grepl("[[a-z][0-9]]", "d") and grepl("[[a-z]]", "d") actually mean?

Forthermore, I know that we need to use double square brackets, say, for things like "[[:digit:]]", because "[:digit:]" would rather search for ":", "d", "i", "g" or "t" (from this question). But how exactly is the structure of "[[:digit:]]" being interpreted? (just a guess: does R interpret it as the trivial union of [:digit:] with itself so that it's just a 'readable' [:digit:] for R?)

user_3832970 · Answer

You should be careful with square brackets inside regular expressions. Now, I will assume you are using the default TRE regex library that is used with the base R regex functions (when no perl=TRUE is passed).

In this case, you should differentiate between

[ and ] that mark the start and end of the POSIX character class, e.g. [:alpha:]
[ and ] that mark the start and end of a bracket expression
unescaped ] that is not preceded with a matching unescaped open [ is treated as a literal ] char.

The [[a-z][0-9]] regex is not equal to [a-z0-9].

[[a-z][0-9]] matches strings like [1], a1] and means:
- [[a-z] - a bracket expression matching a [ char or any lowercase ASCII letter
- [0-9] - a digit
- ] - a ] char.

The [a-z0-9] bracket expression just matches a lowercase ASCII letter or digit.

There is no such a construct in regex as double square brackets. Inside a character class, [ can be used anywhere to match a [ char. ] only matches a ] when it is placed at the start of a bracket expression:

[a-z[] matches a single char, a lowercase ASCII letter or [
[][a-z] matches a single char, a lowercase ASCII letter, [ or ]
[[a-z]] matches a lowercase ASCII letter or [ and then a ] char (so, 2 chars in total)

More things to consider

[:digit:] indeed matches :, d, i, g or t in TRE and PCRE regex, but ICU regex library (used in stringr/stringi regex functions) supports these bare POSIX character classes (as mentioned in POSIX character class does not work in base R regex) and matches a digit even outside bracket expressions
[A-z] matches more than just letters.

Problem with the outputs from using double square brackets

1 Answers