Rust: Unicode-aware string matching

user_3491462 0
Viewed 329

I want to determine if some string contains a certain substring while respecting combining characters. To illustrate the problem, consider the following example in Rust:

fn main() {
    let a_umlaut = "a\u{0308}"; // "ä"
    println!("{}", a_umlaut.starts_with("a")); // true
}

Basically, the above shows that "ä".starts_with("a") is true (note the diaeresis over the first "a"). I do understand the reason for this behavior on a technical level, but I still want the above code to output false, since "ä" and "a" are two different user-perceived characters.

Is there an existing function/create that performs string matching while respecting combining characters?

rust unicode
1 Answers

I expanded out my idea from my comment. This regular expression will match at the beginning of a string, an a character without an umlaut.

use regex::Regex;

fn main() {
    let a_umlaut = "a\u{0308}"; // "ä"
    println!("Original string: {}", a_umlaut);
    println!("Start with regular 'a': {}", a_umlaut.starts_with("a")); // true

    let re = Regex::new(r"^a[^\u{0308}]").unwrap(); // Matches non-combined "a" at the front
    tester(&re, a_umlaut);      // "a" with umlaut behind
    tester(&re, "blessed are"); // "a" in the middle, not the front
    tester(&re, "amore!");      // "a" at the front

}

fn tester(re: &Regex, test: &str)
{
    println!("For string: '{}' with Regex: '{}', match is: {}", test, re.as_str(), re.is_match(test));
}

Output:

Original string: ä
Start with regular 'a': true
For string: 'ä' with Regex: '^a[^\u{0308}]', match is: false
For string: 'blessed are' with Regex: '^a[^\u{0308}]', match is: false
For string: 'amore!' with Regex: '^a[^\u{0308}]', match is: true

Playground link

The idea here is that you could expand the list of characters you do not want to match for inside the regex, so anything that would combine with a would also be listed. This does run into the problem that this could be a very long list, but if this is a constrained problem, this approach could work.

user_838390
Related