spaCy use only some components

user_10011330 0
Viewed 315

I am using spaCy for my project. It works magnificiently, only it is a bit time-consuming. I am looking for ways to reduce processing time. I have realized that calling nlp on my text will perform many operations: tokenization, ner, ... (doc here: https://spacy.io/usage/spacy-101#pipelines) ; while in some parts of my code, I only need to perform e.g. vectorization. Is it possible to apply only some components of the pipeline to reduce processing time?

spacy
1 Answers

It is possible to disable modules and enable them back when necessary. When speeding up really is an issue, try using the pipe functionality, this speeds up for loads of documents.

    nlp = spacy.load("en_core_web_sm")
    for doc in nlp.pipe(texts, disable=["tagger", "parser"]):
        print([(ent.text, ent.label_) for ent in doc.ents])

Source

user_67011630
Related