Convert Pandas dataframe to Sparse Numpy Matrix directly

user_213216 0
Viewed 68043

I am creating a matrix from a Pandas dataframe as follows:

dense_matrix = np.array(df.as_matrix(columns = None), dtype=bool).astype(np.int)

And then into a sparse matrix with:

sparse_matrix = scipy.sparse.csr_matrix(dense_matrix)

Is there any way to go from a df straight to a sparse matrix?

Thanks in advance.

pandas scipy numpy python
3 Answers

Solution:

import pandas as pd
import scipy
from scipy.sparse import csr_matrix

csr_matrix = csr_matrix(df.astype(pd.SparseDtype("float64",0)).sparse.to_coo())

Explanation:

to_coo needs the pd.DataFrame to be in a sparse format, so the dataframe will need to be converted to a sparse datatype: df.astype(pd.SparseDtype("float64",0))

After it is converted to a COO matrix, it can be converted to a CSR matrix.

user_193528930

There is a way to do it without converting to dense en route: csr_sparse_matrix = df.sparse.to_coo().tocsr()

user_54776670
Related