We have sparse csr matrix and while slicing the matrix we are getting out of memory (OOM) issue.
X[:, feature_sample]
Here X is of the dimension : [8000000, 138106]
feature_sample is a randomly generated boolean array of the shape (138106,) Ex of it : [True True False False ......True]
Here since column slicing is being done csc matrix is more efficient for the operation but modifying X to a csc matrix didn't help address the OOM issue.
Tried converting this to a dask array but before proceeding need to convert it back to a csr matrix for the next set of operations on X so that resulted in failure.
Looking for suggestions that can help avoid the OOM while slicing the matrix.