I am trying to compute the .95 quantile over the 3rd dimension of 3D NumPy array using NumPy's quantile function. I am finding that this is taking an extremely long time to compute and is creating a major bottleneck in my analysis. The following code appears to reproduce the lengthy computation I am experiencing:
import numpy as np
arr = np.random.normal(1, 3, size=(2000, 2000,200))
quants_95 = np.quantile(arr, q = .95, axis = 2)
Perhaps one of the reasons this seems so much slower than the rest of my analysis is that np.quantile appears to be running on a single core whereas other numpy functions (such as np.correff) run on multiple cores as default. Does anyone know of a way to run np.quantile on multiple cores, or a similar function that does?