\s doesn't actually capture all whitespace characters

user_1490169 0
Viewed 891

In my Java 8 app, I am scanning for whitespaces in text passed in. But \s in my Regular Expression doesn't capture all whitespaces. The one whitespace that I've found that it doesn't capture so far in my testing is Non-breaking Space (Unicode 00A0). This was my regular expression that was running into that issue:

Pattern p = Pattern.compile("\\s");

To solve this, I added \h to my Regular Expression:

Pattern p = Pattern.compile("[\\s\\h]");

Now, are there any other whitespaces that I need to be aware of that wont be captured by \s\h?

java-8 unicode regex java
3 Answers

By default, \s only matches ASCII whitespace characters ([ \t\n\x0B\f\r]). There are two ways to overcome this limitation

  1. Use Unicode character properties: Pattern.compile("\\p{IsWhiteSpace}")

  2. Make the predefined character class use Unicode properties:
    Pattern.compile("\\s", Pattern.UNICODE_CHARACTER_CLASS)
    This can also be enabled via the embedded flag (?U)

Pattern[] pattern = {
    Pattern.compile("\\s"),
    Pattern.compile("\\s", Pattern.UNICODE_CHARACTER_CLASS),
    Pattern.compile("((?U)\\s)"),
    Pattern.compile("\\p{IsWhiteSpace}")
};
String s = " \t\n\u00A0\u2002\u2003\u2006\u202F";
for(Pattern p: pattern) {
    int count = 0;
    for(Matcher m = p.matcher(s); m.find(); ) count++;
    System.out.printf("%-19s: %d matches%n",
      p.pattern()+((p.flags()&Pattern.UNICODE_CHARACTER_CLASS)!=0? " [(?U) via flags]": ""),
      count);
}

\s                 : 3 matches
\s [(?U) via flags]: 8 matches
((?U)\s)           : 8 matches
\p{IsWhiteSpace}   : 8 matches
user_27114880

According to the Pattern class documentation the characters that match \s are \t\n\x0B\f\r.

However, Unicode indeed supports a whole lot more space characters. Examples include:

  • \u2002: En space
  • \u2003: Em space
  • \u2003: Thin space
  • \u202F: Narrow no-break space
user_115957280

We use guava for this...

String s = " \t\n\u00A0\u2002\u2003\u2006\u202F";
int b = CharMatcher.whitespace().countIn(s);
System.out.println(b); // 8
user_10593720
Related