how to extract specific words in negspacy

user_7965340 0
Viewed 941

When I apply negspacy to my sentence, I want negspacy to consider specific phrase as a single entity and give me the output for it.

import en_core_sci_lg
from negspacy.negation import Negex
nlp = en_core_sci_lg.load()

negex = Negex(nlp, language = "en_clinical_sensitive")
nlp.add_pipe(negex, last=True)

doc = nlp(""" patient has no signs of shortness of breath. """)

for word in doc.ents:
    print(word, word._.negex)

The output is -

patient False
shortness True

I want the output to be -

patient False
shortness of breath True

How can I consider phrases like "shortness of breath", "sore throat", "respiratory distress" as a single entity.

I have tried -

import en_core_sci_lg
from negspacy.negation import Negex
nlp = en_core_sci_lg.load()
from spacy.pipeline import EntityRuler
ruler = EntityRuler(nlp)
patterns =  [{"label": "ENTITY", "pattern": [{"LOWER": "shortness"}, {"LOWER": "of"}, {"LOWER": "breath"}]}]

ruler.add_patterns(patterns)
nlp.add_pipe(ruler)
negex = Negex(nlp, language = "en_clinical")
nlp.add_pipe(negex, last=True)

doc = nlp("""patient has no signs of shortness of breath. """)

for word in doc.ents:
    print(word, word._.negex)

The output is still coming -

patient False
shortness True

What can I do to solve this problem

spacy nlp python
3 Answers

You may be glad to know the solution to your problem is very simple. You are just missing the keyword argument overwrite_ents=True in the EntityRuler constructor, so the custom pattern you are adding is being overwritten by other entities. Just change:

ruler = EntityRuler(nlp)

to

ruler = EntityRuler(nlp, overwrite_ents=True)

Now, my output is:

patient False
shortness of breath True
user_106254200

You can just use:

nlp.add_pipe(ruler, before="ner")

This Should solve the issue that you have

user_141841730
user_79653400
Related