AtttributeError with `vocab` in Migratation from Gensim 3 to 4 for NLP

Manwest 0
Viewed 37

I have the following problem to migrate Gensim 3 to 4. The code is:

from gensim.models import Word2Vec
from sklearn.decomposition import PCA
from matplotlib import pyplot
import numpy as np
# define training data
sentences = [['this', 'is', 'the', 'first', 'sentence', 'for', 'word2vec'],
            ['this', 'is', 'the', 'second', 'sentence'],
            ['yet', 'another', 'sentence'],
            ['one', 'more', 'sentence'],
            ['and', 'the', 'final', 'sentence']]
# train model
model = Word2Vec(sentences, min_count=1)
# fit a 2d PCA model to the vectors
X = model[model.wv.vocab]
pca = PCA(n_components=2)
result = pca.fit_transform(X)
# create a scatter plot of the projection
pyplot.scatter(result[:, 0], result[:, 1])
words = list(model.wv.vocab)
for i, word in enumerate(words):
    pyplot.annotate(word, xy=(result[i, 0], result[i, 1]))
pyplot.show()

The error is:


    ---> 14 X = model[model.wv.vocab]

The result of this code must be: pyplot

gensim nlp python
2 Answers

If you search Google for [migrate Gensim 3 to 4], the top result should be the project's help wiki page, "Migrating from Gensim 3.x to 4".

The 4th item on that page points out that the vocab attribute has changed, with examples of how to update older code that will no longer work (commented with ) into working code (commented with ):

4. vocab dict became key_to_index for looking up a key's integer index, or get_vecattr() and set_vecattr() for other per-key attributes:

rock_idx = model.wv.vocab["rock"].index  #    
rock_cnt = model.wv.vocab["rock"].count  # 
vocab_len = len(model.wv.vocab)  # 

rock_idx = model.wv.key_to_index["rock"]
rock_cnt = model.wv.get_vecattr("rock", "count")  # 
vocab_len = len(model.wv)  #

Your code only uses model.wv.vocab as a way to get a sequence or list of the model's known words, so you could just replace all instances of model.wv.vocab with model.wv.key_to_index, which just like the old .vocab is a dict with all known words as keys. EG:

X = model[model.wv.index_to_key]

...and later also...

words = list(model.wv.index_to_key)
gojomo0

I found a solution:

    ...
    X = model.wv[model.wv.key_to_index]
    ...
    words = list(model.wv.index_to_key)
    ...

Note: Solution provided by OP on question section.

4b00
Related