Remove a word in a span from SpaCy?

user_5942350 0
Viewed 3290

I am parsing a sentence with Spacy like following:

import spacy
nlp = spacy.load("en")
span = nlp("This is some text.")

I am wondering if there is a way to delete a word in the span, while still keep the remaining words format like a sentence. Such as 

del span[3]

which could yield a sentence like

This is some.

If some other methods without SpaCy could achieve the same effect that will be great too.

spacy nlp python-3.x
2 Answers

There is a workaround for that.

The idea is that you create a numpy array from the doc, you delete the entry you don't want and then you create a doc from the new numpy array.

import spacy
from spacy.attrs import LOWER, POS, ENT_TYPE, IS_ALPHA
from spacy.tokens import Doc
import numpy

def remove_span(doc, index):
    np_array = doc.to_array([LOWER, POS, ENT_TYPE, IS_ALPHA])
    np_array_2 = numpy.delete(np_array, (index), axis = 0)
    doc2 = Doc(doc.vocab, words=[t.text for i, t in enumerate(doc) if i!=index])
    doc2.from_array([LOWER, POS, ENT_TYPE, IS_ALPHA], np_array_2)
    return doc2

# load english model
nlp = spacy.load('en')
doc = nlp("This is some text")
new_doc = remove_span(doc, 3)
print(new_doc)

Hope it helps!

user_60822750

The other answer requires you to lose POS information.

def remove_i_element_from_span(span, index):
  nlp_list = list(span)
  del nlp_list[index]
  return nlp(" ".join([e.text for e in nlp_list]))
user_96487640
Related