Using custom Token extensions in spaCy's Matcher

Admin 1
Viewed 609

I just added the following extension to Token in spaCy:

from spacy.tokens import Token
has_dep = lambda token,name: name in [child.dep_ for child in token.children]
Token.set_extension('HAS_DEP', method=has_dep)

So, I want to check if a token has a certain specified dependency name as one of its children, so the following:

doc = nlp(u'We are walking around.')
walking = doc[2]
walking._.HAS_DEP('nsubj')

Outputs True, because 'walking' has a child whose dependency tag is 'nsubj' (i.e. the word 'we').

However, I don't understand how to use this extension with spaCy's Matcher. Below is what I've written. The output I expect is walking, but it doesn't seem to work:

matcher = Matcher(nlp.vocab)

pattern = [
    {"_": {"HAS_DEP": {'name': 'nsubj'}}}  # this is the line I'm not sure of
    ]

matcher.add("depnsubj", None, pattern)

doc = nlp("We're walking around the house.")
matches = matcher(doc)

for match_id, start, end in matches:
    string_id = nlp.vocab.strings[match_id]  
    span = doc[start:end]
    print(span)
spacy nlp methods matcher python
2 Answers

I think your goal may be rather achieved with a getter:

import spacy
from spacy.matcher import Matcher
from spacy.tokens import Token
has_dep = lambda token: 'nsubj' in [child.dep_ for child in token.children]
Token.set_extension('HAS_DEP_NSUBJ', getter=has_dep, force=True)

nlp = spacy.load("en_core_web_md")
matcher = Matcher(nlp.vocab)
matcher.add("depnsubj", None, [{"_": {"HAS_DEP_NSUBJ": True}}])

doc = nlp("We're walking around the house.")
matches = matcher(doc)

for match_id, start, end in matches:
    string_id = nlp.vocab.strings[match_id]  
    span = doc[start:end]
    print(span)

walking
user_43170580

I think you can use doc.retokenize() and token.head instead as below:

from spacy.matcher import Matcher
import en_core_web_sm

nlp = en_core_web_sm.load()

matcher = Matcher(nlp.vocab)
pattern = [{'DEP': 'nsubj'}]
matcher.add("depnsubj", None, pattern)

doc = nlp("We're walking around the house.")
matches = matcher(doc)

matched_spans = []
for match_id, start, end in matches:
    span = doc[start:end]
    matched_spans.append(doc[start:end])

matched_tokens = []
with doc.retokenize() as retokenizer:
    for span in spans:
        retokenizer.merge(span)
        for token in span:
            print(token.head)

Output:

walking
user_111472950
Related