I have a large structured numpy array with the following datatype:
> my_array.dtype
= dtype([('field1', '<i4', (32,)), ('field2', '<i4', (425,)),
('field3', '<i4', (8021,))])
My goal is to access any given single element as fast as possible. If I slice my array by a field name, Numba is able to perform in non-python-mode.
@numba.njit
def test_function(my_array_sliced):
for i in range(len(my_array_sliced)):
_ = my_array_sliced[i]
return
> %timeit test_function(my_array['field1'])
399 ns ± 10.8 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
This works well, but I realized that I'm able to achieve better results performance-wise by accessing my array a bit differently with a 'constant' field name:
@numba.njit
def test_function2(my_array):
for i in range(len(my_array)):
_ = my_array[i]['field1']
return
> %timeit test_function2(my_array)
280 ns ± 5.88 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
399ns vs 280ns - that's a big difference! I can't really tell why there a such a difference.
If I try to get the better performance by adding a parameter to my function, I'll get a typing error.
@numba.njit
def test_function3(my_array, v):
for i in range(len(my_array)):
_ = my_array[i][v]
return
> %timeit test_function3(my_array, 0):
TypingError: Failed at nopython (nopython frontend)
Invalid usage of getitem with parameters (Record([('field1', '<i4',
(32,)), ('field2', '<i4', (425,)), ('field3', '<i4', (8021,))]), int64)
Similar thing happens if I replace the int-parameter with a string parameter such as 'field1' (even tho, I shouldn't be using strings in the first place, because Numba doesn't really deal with them).
So performance wise, currently the best bet is to create different functions for every field name. That's obviously insane. How should I modify my code to achieve the best performance possible?
Thank you!