How to use parallel processing to run a for loop and some functions?

user_12934547 0
Viewed 50

I have a an algorithm that has several for loops and some functions that sometimes need to be run in sequence or sometimes can be run in parallel. I am providing a pseudo-code and an example below.

Here, m is a mathematical model and I am trying to variables to the model m. The for loops are independent of each other (I have several such for loops not 2).

for i in range(1,N+1):
    for d in range(1,delta+1):
        for t in range(1,T+1):
            for k in range(1,K+1):
                z[(i,d,t,k)] = m.addVar(vtype = GRB.BINARY, name="z%d,%d,%d,%d" % (i,d,t,k))

for k in range(1,K+1):
    for d in range(1,delta+1):
        Q[(k,d)] = m.addVar(vtype = GRB.BINARY, name="Q%d,%d" % (k,d))

Once the model m is built completely i.e. all the for loops are completed I have the command that solves the optimization problem. This can only be done after the model is built completely. So the next command is:

z,Q = Solve(m)

Next, I am using other for loops to copy the results from model m. These cannot be used directly and must be copied in the way I have used.

for i in range(1,N+1):
    for d in range(1,delta+1):
        for t in range(1,T+1):
            for k in range(1,K+1):
                z_value[(i,d,t,k)] = z[(i,d,t,k)].X

for k in range(1,K+1):
    for d in range(1,delta+1):
        Q_value[(i,inst)] = Q[(k,d)].X

This portion is also independent of each other. I have more than two loops to run.

Is there a way that I can use parallel processing for these parts of my code. How do I do that?

python-multiprocessing parallel-processing multiprocessing python
1 Answers

You could reduce the nested loops to one loop using itertools.product for instance your first loop can be simplified to the below example using no nesting:

from itertools import product

for idtk in product(range(1,N+1), range(1,delta+1), range(1,T+1), range(1,K+1)):
    #idtk is a tuple the same as (i, d, t, k)
    z[idtk] = m.addVar(vtype = GRB.BINARY, name="z%d,%d,%d,%d" % idtk)
user_2250200
Related