I have a list called corpus that I am attempting TF-IDF on, using the sklearn in-built function. The list has 5 items. Each of these items comes from text files.
I have generated a toy list called corpus for this example.
corpus = ['Hi what are you accepting here do you accept me',
'What are you thinking about getting today',
'Give me your password to get accepted into this school',
'The man went to the tree to get his sword back',
'go away to a far away place in a foreign land']
vectorizer = TfidfVectorizer(stop_words='english')
vecs = vectorizer.fit_transform(corpus)
feature_names = vectorizer.get_feature_names()
dense = vecs.todense()
lst1 = dense.tolist()
df = pd.DataFrame(lst1, columns=feature_names)
df
Using the above code, I was able to get a dataframe with 5 rows (for each item in the list) and n-columns with the tf-idf for each term in this corpus.
As a next step, I want to build the word cloud with largest tf-idf terms across the 5 items in the corpus getting the highest weight.
I tried the following:
x = vectorizer.vocabulary_
Cloud = WordCloud(background_color="white", max_words=50).generate_from_frequencies(x)
This clearly does not work. The dictionary is a list of words with an index attached to it, not a word scoring.
Hence, I need a dictionary that assigns the TF-IDF score to each word across the corpus. Then, the word cloud generated has the highest scored words as the largest size.