I am trying to save my tf idf vectorizer with pickle, but unfortunatelly i keep getting an error
Can't pickle local object 'preprocess.<locals>.my_tokenizer'
This is the vectorizer where my tokenizer is my custom tokenizer :
tfidf_vectorizer = TfidfVectorizer(tokenizer= my_tokenizer,
ngram_range=(1,3), min_df=0.005, max_df=0.2)
This is the pickle
import pickle
with open('vectorizer.game', 'wb') as fin:
pickle.dump(tfidf_vectorizer, fin)
I do not know how to make it global