Python: spacy add stop words issue

user_8171986 0
Viewed 110

I am trying to add some words to my stop_words in spacy to built then a world cloud but somehow these words still appear in my word cloud. I see that words are taken out but not the ones I added?

from wordcloud import WordCloud, STOPWORDS
import matplotlib.pyplot as plt
import extract_msg
import os
import spacy

nlp = spacy.load("de_core_news_sm")
nlp.Defaults.stop_words |= {"apple","strawberry"}

text_reduced_empty_spaces = “apple, strawberry, banana, peach“
doc = nlp(text_reduced_empty_spaces)
lemmatisiert = ' '.join([token.lemma_ for token in doc])
text_lemmatisiert = nlp(lemmatisiert)
cleaned = ' '.join([token.text for token in text_lemmatisiert if not token.is_stop and not token.is_punct])


wordcloud = WordCloud(background_color="white",width=1920, height=1080).generate(cleaned)
plt.imshow(wordcloud, interpolation="bilinear")
plt.axis("off")
plt.show()
spacy python
1 Answers

It seems that the problem is larger and lower cases. If u use a capital letter as the first letter to add into the STOP_WORDS it is not picked up and taken out later. Only if you use lower case words to add to the STOP_WORDS list it works.

user_81719860
Related