Is the following thread safe in spacy?
nlp = spacy.load('en')
nlp(text)
1 Answers
Yes. See this thread (no pun intended...) for more information: https://github.com/explosion/spaCy/issues/172
If possible, instead of multi-threading the call to nlp, consider calling nlp.pipe which is a multithreaded call to the spaCy pipeline.
for doc in nlp.pipe(texts, n_threads=16, batch_size=10000):
analyse_text(doc)
More info here: https://explosion.ai/blog/multithreading-with-cython
Related