How can I use SpaCy Matcher (or PhraseMatcher) class for the extracting the sequence of 2 items?

user_9349225 0
Viewed 1481

I have the following task: extract combinations of two tokens from a text. Each of them belongs to the list. For example:

colors=['red','gray','black','white','brown']
animals=['fox','bear','hare','squirrel','wolf']

In the SpaCy docs https://spacy.io/usage/rule-based-matching it is described pretty good how to match all of these words from the both lists or how to match sequence of the particular two, e.g.:

pattern = [{"LOWER": "red"}, {"LOWER": "fox"}]

But I need to match any combination like "red squirrel" or "white bear". Can I achieve this by means of Matcher (or PhraseMatcher) in SpaCy or i need to use any additional python modules? Do someone have an idea of illegant solving?

Thanks in advance for any help.

spacy nlp matcher python
1 Answers

You may use regex in SpaCy, too. For example, fr"(?i)^({'|'.join(colors)})$" will create a pattern that matches a token in a case insensitive way that will match any one of the colors items.

import spacy
from spacy.matcher import Matcher

nlp = spacy.load("en_core_web_sm")

matcher = Matcher(nlp.vocab)

colors=['red','gray','black','white','brown']
animals=['fox','bear','hare','squirrel','wolf']
pattern = [
   {'TEXT': {"REGEX": fr"(?i)^({'|'.join(colors)})$"}},
   {'TEXT': {"REGEX": fr"(?i)^({'|'.join(animals)})$"}}
]
matcher.add("ColoredAnimals", None, pattern)

doc = nlp("Hello, red fox! Hello Black Hare! What's up whItE sQuirrel, brown wolf and gray bear!")
matches = matcher(doc)
for match_id, start, end in matches:
    string_id = nlp.vocab.strings[match_id]
    span = doc[start:end]
    print(match_id, string_id, start, end, span.text)

Output:

8757348013401056599 ColoredAnimals 2 4 red fox
8757348013401056599 ColoredAnimals 6 8 Black Hare
8757348013401056599 ColoredAnimals 12 14 whItE sQuirrel
8757348013401056599 ColoredAnimals 15 17 brown wolf
8757348013401056599 ColoredAnimals 18 20 gray bear

You may directly extract phrases with regex:

import re
colors=['red','gray','black','white','brown']
animals=['fox','bear','hare','squirrel','wolf']
pattern = fr"(?i)\b(?:{'|'.join(colors)})\s+(?:{'|'.join(animals)})\b"
doc_string = "Hello, red fox! Hello Black Hare! What's up whItE sQuirrel, brown wolf and gray bear!"
print ( re.findall(pattern, doc_string) )
# => ['red fox', 'Black Hare', 'whItE sQuirrel', 'brown wolf', 'gray bear']

See the Python demo

Here, non-capturing groups are used in order not to create additional items in the resulting list, \s+ matches 1 or more whitespace chars, and \b are used as word boundaries instead of ^ (start of string) and $ (end of string) anchors.

user_38329700
Related