Inverse X.toarray into a CountVectorizer in sklearn

user_1052117 0
Viewed 196

I'm following documentation here:

https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html

>>> from sklearn.feature_extraction.text import CountVectorizer
>>> corpus = [
...     'This is the first document.',
...     'This document is the second document.',
...     'And this is the third one.',
...     'Is this the first document?',
... ]
>>> vectorizer = CountVectorizer()
>>> X = vectorizer.fit_transform(corpus)
>>> print(vectorizer.get_feature_names())
['and', 'document', 'first', 'is', 'one', 'second', 'the', 'third', 'this']
>>> print(X.toarray())
[[0 1 1 1 0 0 1 0 1]
 [0 2 0 1 0 1 1 0 1]
 [1 0 0 1 1 0 1 1 1]
 [0 1 1 1 0 0 1 0 1]]

Suppose I already have a term frequency matrix like the one given in X.toarray(), but I didn't use CountVectorizer to obtain it.

I want to apply a TfIDF to this matrix. Is there a way for me to take a count array + a dictionary and apply some inverse of this function as a constructor to get a fit_transformed X?

I'm looking for...

>>> print(X.toarray())
[[0 1 1 1 0 0 1 0 1]
 [0 2 0 1 0 1 1 0 1]
 [1 0 0 1 1 0 1 1 1]
 [0 1 1 1 0 0 1 0 1]]


>>> V = CountVectorizerConstructorPrime(array=(X.toarray()), 
                                        vocabulary=['and', 'document', 'first', 'is', 'one', 'second', 'the', 'third', 'this'])

such that:

>>> V == X
True
scikit-learn
1 Answers

The X constructed by the CountVectorizer is a sparse matrix in SciPy's compressed sparse row (csr) format. So you can construct it directly from any word count matrix with the appropriate SciPy function:

from scipy.sparse import csr_matrix

V = csr_matrix(X.toarray())

Now V and X are equal, although this may not be obvious, because V == X will give you another sparse matrix (or rather complain that the matrix is not sparse despite the intended format, see this question). But you can check it like this:

(V != X).toarray().any()

False

Note that the word list was not needed, because the matrix only encodes the frequencies of all distinct words, no matter what they are.

user_130141720
Related