How to perform efficient sparse matrix multiplication by using tf.matmul?

I'm trying to perform a sparse matrix multiplication by using tf.matmul().

However, the inference speed is much more slower than dense matrix multiplication.

According to the description in tf.sparse_matmul() :

• The breakeven for using this versus a dense matrix multiply on one platform was 30% zero values in the sparse matrix.

Thus , I make the sparse matrix with 7/8 zero values.

Here is my code:

import tensorflow as tf
import numpy as np
import time
a = tf.Variable(np.arange(1000).reshape(250,4) ,dtype=tf.float32) #dense matrix
b = tf.Variable(np.array([0,0,0,0,0,0,0,1],dtype=np.float32).reshape(4,2),dtype=tf.float32) # sparse matrix
c = tf.matmul(a,b,b_is_sparse=True) # do the sparse matrix multiplication

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    num_iteration = 5000
    num_burnin = 50
    duration = 0

    for i in range(num_iteration+num_burnin):
        startTime  = time.time()
        result = sess.run(c)
        endTime = time.time()
        if i > num_burnin :
            duration+= endTime-startTime

   print(" Average Inference Time = %.3f  ms"%(duration*1000/num_iteration))

I set "b_is_sparse=True" to do a sparse matrix multiplication , and it takes about 0.380 ms on my GeForce GTX 960M.

However , if I set "b_is_sparse=False" to do a dense matrix multiplication , it takes about 0.280 ms.

I have tried to use tf.sparse_tensor_dense_matmul and tf.embedding_lookup_sparse to perform sparse matrix multiplication , but the inference speed is still slower than dense matrix multiplication.

Is there something wrong in my code or other way to perform sparse matrix multiplication ?

Any advice will be greatly appreciated!!