How can I efficiently convert a scipy sparse matrix into a sympy sparse matrix?

Viewed 533

I have a matrix A with the following properties.

<1047x1047 sparse matrix of type '<class 'numpy.float64'>'
    with 888344 stored elements in Compressed Sparse Column format>

A has this content.

array([[ 1.00000000e+00, -5.85786642e-17, -3.97082034e-17, ...,
         0.00000000e+00,  0.00000000e+00,  0.00000000e+00],
       [ 6.82195979e-17,  1.00000000e+00, -4.11166786e-17, ...,
         0.00000000e+00,  0.00000000e+00,  0.00000000e+00],
       [-4.98202332e-17,  1.13957868e-17,  1.00000000e+00, ...,
         0.00000000e+00,  0.00000000e+00,  0.00000000e+00],
       ...,
       [ 4.56847824e-15,  1.32261454e-14, -7.22890998e-15, ...,
         1.00000000e+00,  0.00000000e+00,  0.00000000e+00],
       [-9.11597396e-15, -2.28796167e-14,  1.26624823e-14, ...,
         0.00000000e+00,  1.00000000e+00,  0.00000000e+00],
       [ 1.80765584e-14,  1.93779820e-14, -1.36520100e-14, ...,
         0.00000000e+00,  0.00000000e+00,  1.00000000e+00]])

Now I'm trying to create a sympy sparse matrix from this scipy sparse matrix.

from sympy.matrices import SparseMatrix
A = SparseMatrix(A)

But I get this error message.

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all().

I am confused because this matrix has no logical entries.

Thanks for any help!

3 Answers

The Error

When you get an error that you don't understand, take a bit of time to look at the traceback. Or at least show it to us!

In [288]: M = sparse.random(5,5,.2, 'csr')                                                           

In [289]: M                                                                                          
Out[289]: 
<5x5 sparse matrix of type '<class 'numpy.float64'>'
    with 5 stored elements in Compressed Sparse Row format>

In [290]: print(M)                                                                                   
  (1, 1)    0.17737340878962138
  (2, 2)    0.12362174819457106
  (2, 3)    0.24324155883057885
  (3, 0)    0.7666429046432961
  (3, 4)    0.21848551209470246

In [291]: SparseMatrix(M)                                                                            
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-291-cca56ea35868> in <module>
----> 1 SparseMatrix(M)

/usr/local/lib/python3.6/dist-packages/sympy/matrices/sparse.py in __new__(cls, *args, **kwargs)
    206             else:
    207                 # handle full matrix forms with _handle_creation_inputs
--> 208                 r, c, _list = Matrix._handle_creation_inputs(*args)
    209                 self.rows = r
    210                 self.cols = c

/usr/local/lib/python3.6/dist-packages/sympy/matrices/matrices.py in _handle_creation_inputs(cls, *args, **kwargs)
   1070                             if 0 in row.shape:
   1071                                 continue
-> 1072                         elif not row:
   1073                             continue
   1074 

/usr/local/lib/python3.6/dist-packages/scipy/sparse/base.py in __bool__(self)
    281             return self.nnz != 0
    282         else:
--> 283             raise ValueError("The truth value of an array with more than one "
    284                              "element is ambiguous. Use a.any() or a.all().")
    285     __nonzero__ = __bool__

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all().

A full understanding requires reading the sympy code, but a cursory look indicates that it's trying to handle your input as "full matrix", and looks at rows. The error isn't the result of you doing logical operations on the entries, but that sympy is doing a logical test on your sparse matrix. It's trying to check if the row is empty (so it can skip it).

SparseMatrix docs may not be the clearest, but most examples either show a dict of points, or a flat array of ALL values plus shape, or a ragged list of lists. I suspect it's trying to treat your matrix that way, looking at it row by row.

But the row of M is itself a sparse matrix:

In [295]: [row for row in M]                                                                         
Out[295]: 
[<1x5 sparse matrix of type '<class 'numpy.float64'>'
    with 0 stored elements in Compressed Sparse Row format>,
 <1x5 sparse matrix of type '<class 'numpy.float64'>'
    with 1 stored elements in Compressed Sparse Row format>,
...]

And trying to check if that row is empty not row produces this error:

In [296]: not [row for row in M][0]                                                                  
...
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all().

So clearly SparseMatrix cannot handle a scipy.sparse matrix as is (at least not in the csr or csc format, and probably not the others. Plus scipy.sparse is not mentioned anywhere in the SparseMatrix docs!

from dense array

Converting the sparse matrix to its dense equivalent does work:

In [297]: M.A                                                                                        
Out[297]: 
array([[0.        , 0.        , 0.        , 0.        , 0.        ],
       [0.        , 0.17737341, 0.        , 0.        , 0.        ],
       [0.        , 0.        , 0.12362175, 0.24324156, 0.        ],
       [0.7666429 , 0.        , 0.        , 0.        , 0.21848551],
       [0.        , 0.        , 0.        , 0.        , 0.        ]])

In [298]: SparseMatrix(M.A)                                                                          
Out[298]: 
⎡        0                  0                  0                  0                  0        ⎤
...⎦

Or a list of lists:

 SparseMatrix(M.A.tolist()) 

from dict

The dok format stores a sparse matrix as a dict, which then can be

In [305]: dict(M.todok())                                                                            
Out[305]: 
{(3, 0): 0.7666429046432961,
 (1, 1): 0.17737340878962138,
 (2, 2): 0.12362174819457106,
 (2, 3): 0.24324155883057885,
 (3, 4): 0.21848551209470246}

Which works fine as an input:

SparseMatrix(5,5,dict(M.todok()))

I don't know what's most efficient. Generally when working with sympy we (or at least I) don't worry about efficiency. Just get it to work is enough. Efficiency is more relevant in numpy/scipy where arrays can be large, and using the fast compiled numpy methods makes a big difference in speed.

Finally - numpy and sympy are not integrated. That applies also to the sparse versions. sympy is built on Python, not numpy. So inputs in the form of lists and dicts makes most sense.

from sympy.matrices import SparseMatrix
import scipy.sparse as sps

A = sps.random(100, 10, format="dok")
B = SparseMatrix(100, 10, dict(A.items()))

From the perspective of someone who likes efficient memory structures this is like staring into the abyss. But it will work.

This is a simplified version of your error.

from scipy import sparse
row = np.array([0, 0, 1, 2, 2, 2])
col = np.array([0, 2, 2, 0, 1, 2])
data = np.array([1, 2, 3, 4, 5, 6])
A = sparse.csc_matrix((data, (row, col)), shape=(3, 3))

So A is a sparse matrix with 6 elements:

<3x3 sparse matrix of type '<class 'numpy.intc'>'
    with 6 stored elements in Compressed Sparse Column format>

Calling SparseMatrix() on it returns the same kind of error that you have. You might like to convert A to numpy array first:

>>> SparseMatrix(A.todense())
Matrix([
[1, 0, 2],
[0, 0, 3],
[4, 5, 6]])
Related