I am trying to calculate the dot product between two large Scipy Sparse matrices (5million x 5million product with 5million x 10000). Both matrices fit in memory and I have a 40 core system. However, the dot product operation of Scipy does not parallelise across any more than 1 core.
Currently it is taking a lot of time to run this simple computation. I have read the Scipy documentation, along with this page: https://scipy.github.io/old-wiki/pages/ParallelProgramming , however, it is nowhere mentioned that the dot operation has a parallel implementation.
I also tried installing OpenMP, BLAS, LAPACK etc. installations of the libraries but none of them seems to provide any parallel processing capabilities for Scipy Sparse matrices. However, I can confirm that Numpy dot product is being parallelised well.
- Can it be confirmed that Sparse matrix product in Scipy is not parallelised?
- Is there any possible way to parallelise this operation to get the product faster than running it on one core?