Transform not found in TfidfVectorizer

user_14514276 0
Viewed 65

I am using TfidfVectorizer for vectorize my data. Firstly I initiates it, then I fit it, dump and when I try to load and do transform function I get this error:

transform not found

and my code looks like this:

vectorizer = TfidfVectorizer(max_features=500, lowercase=True, analyzer='word', ngram_range=(1, 5))

X = vectorizer.fit_transform(df['content'])
joblib.dump(X, "./vectorizer.pkl")

vectorizer = joblib.load("./vectorizer.pkl")
# vectorizer here is scipy.sparse._csr.csr_matrix type
X = vectorizer.transform(df['content'])

What is wrong here?

FIXED

vectorizer = TfidfVectorizer(max_features=500, lowercase=True, analyzer='word', ngram_range=(1, 5))

X = vectorizer.fit_transform(df['content'])
joblib.dump(vectorizer, "./vectorizer.pkl")

vectorizer = joblib.load("./vectorizer.pkl")
X = vectorizer.transform(df['content'])
joblib scikit-learn tf-idf vectorization python
1 Answers

The problem is vectorizer = joblib.load("./vectorizer.pkl"). You are overwriting the vectorizer therefore it no longer has transform method.

user_188927490
Related