How determine if a token is part of an entity within Spacy?

user_5144885 0
Viewed 1889

I have

import spacy
nlp = spacy.load("en_core_web_lg")
line = "Rio de Janeiro is the capital of.."
doc = nlp(line)
for tok in doc:
    printf(tok.lemma_)
for ent in doc.ents:
    printf(e.lemma_)

I want obtain wikization: "[[Rio de Janeiro]] [[be|is]] [[the]] [[capital]] [[of]].." how determine if token "Rio" is part of entity "Rio de Janeiro"?

spacy python
2 Answers

Use the ent_type or ent_type_ attribute, if the value is not an empty string it is an entity.

Edit: for attribute ent_iob or ent_iob_ “B” means the token begins an entity, “I” means it is inside an entity, “O” means it is outside an entity, and "" means no entity tag is set.

 import spacy
    nlp = spacy.load("en_core_web_lg")
    line = "Rio de Janeiro is the capital of.."
    doc = nlp(line)
    for tok in doc:
        print(tok, tok.ent_type_, tok.ent_iob_)

Output:

Rio GPE B
de GPE I
Janeiro GPE I
is  O
the  O
capital  O
of  O
..  O
user_140563970

Entities have start and end property: indicies of token stream. I can write:

import spacy
nlp = spacy.load("en_core_web_lg")
line = "Rio de Janeiro is the capital of.."
doc = nlp(line)
if len(doc.ents)>0:
    firstEnt = doc.ents[0].start
else:
    firstEnt = len(doc)
for j in range(firstEnt):
    print(doc[j])
j = firstEnt
for i in range(len(doc.ents)):
    ent = doc.ents[i]
    while j<ent.start:
        print(doc[j])
        j+=1
    print(ent)
if len(doc.ents) > 0:
    j = ent.end
while j<len(doc):
    print(doc[j])
    j+=1
user_51448850
Related