I have a function of parameters I am fitting to data (and so is called many times). the function could be thought of as F = conv(R1,T1) + conv(R2,T2) + conv(R3,T3).
It's not especially important for this what R and T are. The main point is I have to do 3 convolutions (which I do with scipy), so I wanted to do these 3 in parallel to optimise the code. The code below shows how I implement it. This is a simplified example, in the real thing the two functions are more complicated, and I will give them as inputs to f, but the actual scipy convolution step will be identical.
In the full working f will be of the form f(T,R), and may well take further args.
`
import time
import numpy as np
from scipy.signal import fftconvolve
from multiprocessing import Pool
q = np.arange(0,5,0.0001)
theory = np.exp(-1*q)
q2 = np.arange(-5,5,0.0001)
resolution = np.exp(-1/2*(q2**2))
def f( i ):
return fftconvolve( theory , resolution , mode='same' ) #time.sleep(1)
start = time.time()
f( 0 )
end = time.time()
print(f'Time 0 = { end-start }')
start = time.time()
for i in range(0,3):
f( i )
end = time.time()
print(f'Time 1 = { end-start }')
pool = Pool( processes= 3 )
start = time.time()
pool.map( f , [0, 1, 2, ])
end = time.time()
print(f'Time 2 = { end-start }')
`
I get:
Time 0 = 0.005838155746459961
Time 1 = 0.01575756072998047
Time 2 = 0.028168439865112305
I would have assumed that Time 2 would be a little over 3 times Time 0. So something is going wrong.
I get times of: 1,3,1 when I use the commented out time.sleep(3) instead of the convolution, which is what I expect.
Can anyone help me figure out how to get this to run in parallel? It may mean using something other Multiprocessing. To be clear what I was aiming for is that the function f is run 3 times (for 3 different set of args) but in parallel, so that the total time is about the same time as just one run of f.