AttributeError: 'CountVectorizer' object has no attribute '_load_specials'

amnpawar 0
Viewed 26

I am dumping my pretrained doc2vec model using below command

model.train(labeled_data,total_examples=model.corpus_count, epochs=model.epochs)    

print("Model Training Done")

#Saving the created model
model.save(project_name + '_doc2vec_vectorizer.npz')

vectorizer=CountVectorizer()

vectorizer.fit(df[0])

vec_file = project_name + '_doc2vec_vectorizer.npz'
**pickle.dump(vectorizer, open(vec_file, 'wb'))**


vdb = db['vectorizers']

and then I am loading Doc2vec model using below command in another function

loaded_vectorizer = pickle.load(open(vectorizer, 'rb'))

and then getting the error CountVectorizer has no attribute _load_specials on below line i.e model2

model2= gensim.models.doc2vec.Doc2Vec.load(vectorizer)

The gensim version being used by me is 3.8.3 as I am using the LabeledSentence class

gensim python-3.x
1 Answers

The .load() method on Gensim model classes should only be used with objects of exactly that same class that were saved to file(s) *using the Gensim .save() method.

Your code shows you trying to use Doc2Vec.load() with the vectorizer object itself (not a file path to the previously-saved model), so the error is to be expected.

If you actually want to pickle-save & then pickle-load the vectorizer object, be sure to:

  • use a different file path than you did for the model, or you'll overwrite the model file!
  • use pickle methods (not Gensim methods) to re-load anything that was pickle-saved
gojomo0
Related