Join all arrays inside a ndarray in numpy

Question

I have an ndarray generated with all the possible combinations of 3 array like this:

countries = ["AF"... "Zw"]
names = ["name1",... "nameN"]
var_type = ['var1', 'var2', 'var3']
combinations = np.array(np.meshgrid(names, var_type,countries)).T.reshape(-1, 3)

which gives an ndarray with the following result:

array([
   ['name1', 'var1', 'AF'],
   ['name1', 'var2', 'AF'],
   ['name1', 'var3', 'AF'],
   ...,
   ['nameN', 'var1', 'ZW'],
   ['nameN', 'var2', 'ZW'],
   ['nameN', 'var3', 'ZW']
])

And I want to join each individual sub-array to get a new array with the merged values like this:

array([
   "name1-var1-AF",
   "name1-var2-AF",
   "name1-var3-AF",
   ...,
   "nameN-var1-ZW",
   "nameN-var2-ZW",
   "nameN-var3-ZW"
])

But the only way I fond in google so far is with a for loop like this:

columns = []

for column in combinations:
   columns.append(str('-'.join(column)))

is there a more vectorized way to do this??

user_901925 · Answer

numpy does not fast compiled code for handling strings - other than the basic array manipulation that applies to any dtype. Even the np.char functions use basic python string methods.

In [12]: countries = ["AF","Zw"] 
    ...: names = ["name1","name2", "nameN"] 
    ...: var_type = ['var1', 'var2', 'var3'] 
    ...: combinations = np.array(np.meshgrid(names, var_type,countries)).T.reshape(-1, 3)   

In [14]: ['-'.join(row) for row in _]                                                                
Out[14]: 
['name1-var1-AF',
 'name1-var2-AF',
 'name1-var3-AF',
 'name2-var1-AF',
 ...
 'nameN-var3-Zw']

This is basically a list operation. Iterating on lists is faster

In [18]: timeit ['-'.join(row) for row in combinations]                                              
62.3 µs ± 113 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
In [19]: timeit ['-'.join(row) for row in combinations.tolist()]                                     
6.55 µs ± 31.9 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
In [20]: %%timeit alist = combinations.tolist() 
    ...: ['-'.join(row) for row in alist] 
    ...:  
    ...:                                                                                             
2.88 µs ± 3.4 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

And if I include the time spent creating combinations:

In [29]: %%timeit 
    ...: combinations = np.array(np.meshgrid(names, var_type,countries)).T.reshape(-1, 3)    
    ...: ['-'.join(row) for row in combinations] 
    ...:  
    ...:                                                                                             
164 µs ± 925 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

On the other hand using itertools.product:

In [30]: timeit ['-'.join(tup) for tup in product(names, var_type, countries)]                       
4.17 µs ± 136 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

This is a case where numpy does not help.

Join all arrays inside a ndarray in numpy

1 Answers