What is the easiest way to match non-ASCII characters in a regex? I would like to match all words individually in an input string, but the language may not be English, so I will need to match things like ü, ö, ß, and ñ. Also, this is in Javascript/jQuery, so any solution will need to apply to that.
8 Answers
This should do it:
[^\x00-\x7F]+
It matches any character which is not contained in the ASCII character set (0-127, i.e. 0x0 to 0x7F).
You can do the same thing with Unicode:
[^\u0000-\u007F]+
For unicode you can look at this 2 resources:
- Code charts list of Unicode ranges
- This tool to create a regex filtered by Unicode block.
The situation with regexes, Unicode, and Javascript sucks. It's ridiculous that programmers should have to rely on external libraries to recognize that "Αλφα" is a word, or even that "é" is a letter.
But so it goes.
This guy has written a good library for handling Unicode in Javascript Regexes:
http://blog.stevenlevithan.com/archives/javascript-regex-and-unicode
The Unicode stuff is a plugin to this regex library:
Here's a post about the Unicode extension:
http://blog.stevenlevithan.com/archives/xregexp-unicode-plugin
And the extension page itself:
Great work but it still bums me out that Javascript is so backwards in this regard.
(He wrote a book for O'Reilly about the topic so it's quite possible that he knows what he's talking about.)
The way he implemented it is by adding tables of characters with certain properties. Then, when you contruct a regex with his library, \p{charclass} gets replaced with [allthecharactersintheclass].
You do the same way as any other character matching, but you use \uXXXX where XXXX is the unicode number of the character.
All Unicode-enabled Regex flavours should have a special character class like \w that match any Unicode letter. Take a look at your specific flavour here.
I had a problem with \p working as expected, so I just used a different strategy like:
([^\t]+)\t
Find anything that is not a tab character until the next tab character... obviously this depends on your search source, but you get the idea. Now I don't have to figure out what unicode characters work and don't work etc.
Related