SpaCy's Matcher Regular Expression not matching string

user_837299 0
Viewed 805

I have the following code and regular expresion specified in my pattern. I am trying to get SpaCy to match 24.51c/kWh However the matched text returned is 24.51c

import spacy
from spacy.matcher import Matcher

nlp = spacy.load("en_core_web_sm")

matcher = Matcher(nlp.vocab)
pattern = [{"TEXT": {"REGEX": "[0-9]+[,.]?[0-9]+[Cc/KWHkwh]+"}}]
matcher.add("Usage", None, pattern)
doc = nlp(u"Peak Usage 409 24.51c/kWh $100.25")
matches = matcher(doc)
for match_id, start, end in matches:
    string_id = nlp.vocab.strings[match_id]
    span = doc[start:end]
    print(match_id, string_id, start, end, span.text)

enter image description here

Using this regular expression works in other editors

enter image description here

I cannot find any SpaCy specific tester online for regular expressions. Is there something that I am missing? Any help appreciated.

Thanks

spacy python
1 Answers

SpaCy uses the regular re library. Your text is tokenized in a such a way that / is separated into a different token, 24.51c/kWh is split into 24.51c, / and kWh.

To correctly match the string you need you may use

pattern = [{"TEXT": {"REGEX": "(?i)[0-9]+(?:[,.][0-9]+)?[ckwh]+"}}, 
           {'ORTH': '/'}, {"LOWER": {"REGEX": "(?i)^[ckwh]+$"}}]
matcher.add("Usage", None, pattern)

Or

pattern = [{'ENT_TYPE': 'CARDINAL'}, 
       {'ORTH': '/'}, {"LOWER": {"REGEX": "(?i)^[lckwh]+$"}}]

where {'ENT_TYPE': 'CARDINAL'} matches "numerals that do not fall under another type."

Full snippet:

import spacy
from spacy.matcher import Matcher

nlp = spacy.load("en_core_web_sm")

matcher = Matcher(nlp.vocab)
pattern = [{"TEXT": {"REGEX": "(?i)[0-9]+(?:[,.][0-9]+)?[ckwh]+"}}, 
           {'ORTH': '/'}, {"LOWER": {"REGEX": "(?i)^[ckwh]+$"}}]
matcher.add("Usage", None, pattern)
doc = nlp("Peak Usage 409 24.51c/kWh $100.25")
matches = matcher(doc)
for match_id, start, end in matches:
    string_id = nlp.vocab.strings[match_id]
    span = doc[start:end]
    print(match_id, string_id, start, end, span.text)

Output:

17478281289916104085 Usage 3 6 24.51c/kWh

So, (?i)[0-9]+(?:[,.][0-9]+)?[ckwh]+ matches 1+ digits, then an optional sequence of a dot or comma and then 1+ digits, and then 1+ c, k, w or h letters in a case insensitive way. Then a / token should follow. Then, a token that is equal to c, k, w or h (case insensitively), too.

user_38329700
Related