For each row in matrix1 and each row in matrix2 do difference element-wise

user_9105549 0
Viewed 67

Hello I'd like to ask for a more performance efficient solution for this problem: given two matrices: one with shape (60000, 748) and second with shape (10000, 748) I'd like to iterate over each row in first matrix and for each row in second matrix. Next I'd like to subtract the rows and sum the differences and save them in a "result" matrix. My primitive version is given below.

val = np.array([[0, 2, 4, 2], [2, 4, 1, 5], [3, 3, 1, 9]])
val2 = np.array([[7, 3, 6, 8], [2, 1, 6, 2]])

result = np.zeros((val.shape[0], val2.shape[0]))
for outer_index, validated in enumerate(val):
    for inner_index, trained in enumerate(val2):
        print(np.absolute(validated - trained).sum(0))
        result[outer_index][inner_index] = np.absolute(validated - trained).sum(0)
return result
numpy-ndarray numpy python
1 Answers

You can add a new axis to the first vector, making it in shape (val.shape[0], 1, val.shape[1]) while the second vector is in its original shape and as long as val.shape[1] and val2.shape[1] are equal the 2 vectors are broadcastable. Use this code:

def diff(val, val2):
    return np.abs(val[:,np.newaxis] - val2).sum(2)

Try this than, which is looping on the smaller vector (not using much more memory):

def diff(val, val2):
    if val.shape[0] < val2.shape[0]:
        val2, val = val, val2
    smaller_vec_size = val2.shape[0]
    bigger_vec_size = val.shape[0]
    k = np.empty((smaller_vec_size,bigger_vec_size))
    for i in range(smaller_vec_size):
        k[i] = np.abs(val - val2[i]).sum(1)
    return k
user_161692390
Related