ani have this function that calculate the total dot product for an array, but using pool map function in python slow it down instead of speed it up:
def parallel_execution(x,y):
data_tuple=list(zip(x,y))
with Pool(NUMCORES) as p:
results_dot=p.starmap(my.dot,data_tuple)
return sum(results_dot)
here is the dot function: `
def dot(a,b):
r=a*b
return r
here is the caller:
NUMDATA=10000000
data_X=np.random.rand(NUMDATA)
data_Y=np.random.rand(NUMDATA)
results_parallel=parallel_execution(data_X,data_Y)
profiling the function it says that the most of the time the function is spending is in thread locking how can i avoid this problem and what is this thread.lock method??
ncalls tottime percall cumtime percall filename:lineno(function)
75 73.793 0.984 73.793 0.984 {method 'acquire' of '_thread.lock' objects}