boosting the speed of relating an array and a dictionary

Admin 1
Viewed 32

I have 3 arrays of

a = np.array([[1], [4], [5], [11], [7]])
b = np.array([[14], [3], [2], [10], [12]])
c = np.array([[6], [13], [15], [8], [9]])

I wanted to merge and sort them ascending in each row to get:

[[ 1  6 14]
 [ 3  4 13]
 [ 2  5 15]
 [ 8 10 11]
 [ 7 9 12]]

and to see from what initial array (a,b,c) each value is picked. So, I used this code:

combined = np.concatenate([a, b, c], axis=1)
names = np.array(['a','b','c'])
L = names[np.argsort(combined)]

and it gives me this result:

[['a' 'c' 'b']
 ['b' 'a' 'c']
 ['b' 'a' 'c']
 ['c' 'b' 'a']
 ['a' 'c' 'b']]

I also have a dictionary of:

Test = np.array(range(101,116)).reshape((5,3), order = 'F')

or

[[101 106 111]
 [102 107 112]
 [103 108 113]
 [104 109 114]
 [105 110 115]]

Dic = {'a':Test[:,0], 'b':Test[:,1], 'c':Test[:,2]}

Now I am looking to relate Dic to L using:

new = []
for i in range(0,3):
    for j in range(0,5):
        
        if L[j,i]=='a':
            H = Dic['a'][j]
            
        elif L[j,i]=='b':
            H = Dic['b'][j]
            
        elif L[j,i]=='c':
            H = Dic['c'][j]
            
        new = np.append(new, H)
        
final = new.reshape((5,3), order = 'F')

to give me the final result of:

[[101. 111. 106.]
 [107. 102. 112.]
 [108. 103. 113.]
 [114. 109. 104.]
 [105. 115. 110.]]

However, this process for my real dataset which is very big takes a few hours. I am looking for a better way to speed up my code.

in another word, I am sorting my second array based on the first one.

first array:[['a' 'c' 'b']
             ['b' 'a' 'c']
             ['b' 'a' 'c']
             ['c' 'b' 'a']
             ['a' 'c' 'b']]

second array: [[101 106 111]
              [102 107 112]
              [103 108 113]
              [104 109 114]
              [105 110 115]]

which 1st, 2nd and 3rd columns corresponds to 'a', 'b' and 'c'

numpy dictionary python
2 Answers

Just encode names with integers instead of letters here (This will just be the result of argsort as well).

names = np.arange(3)

Then with take_along_axis:

np.take_along_axis(Test, L, 1)

array([[101, 111, 106],
       [107, 102, 112],
       [108, 103, 113],
       [114, 109, 104],
       [105, 115, 110]])
user_34832030

Loops over dictionaries are always going to be slow. You should try avoiding the dictionary, and use numpy broadcasting instead. For example:

import numpy as np

a = np.array([[1], [4], [5], [11], [7]])
b = np.array([[14], [3], [2], [10], [12]])
c = np.array([[6], [13], [15], [8], [9]])

combined = np.concatenate([a, b, c], axis=1)
i = np.argsort(combined)

Test = np.array(range(101,116)).reshape((5,3), order = 'F')

print(Test[np.arange(5)[:, None], i])
# array([[101, 111, 106],
#        [107, 102, 112],
#        [108, 103, 113],
#        [114, 109, 104],
#        [105, 115, 110]])
user_29378310
Related