Multiprocess a list of commands

Viewed 293

Im trying to run a list of commands. The problem is that the list can be very long, so having multiple commands run simultaneously would be great.

How do I do this with the multiprocessing module?

list_of_commands = [cmd foo, cmd bar, ...]

main_log_file = open( os.getcwd() + '/Error.log', 'w+')

Count = 0
for Job in list_of_commands:
    Count += 1
    child = subprocess.Popen(Job, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    streamdata = child.communicate()[0]
    errcode = child.returncode
    if errcode == 0:
        print ( 'Job', Count, 'Success' )
    elif errcode == 1:
        print ( 'Job', Count, 'Completed With Errors' )
    elif errcode == 2:
        print ( 'Job', Count, 'Error' )
        main_log_file.write ( streamdata.decode('ascii') + str(errcode) + '\n' )

main_log_file.close()

Order of execution doesn't matter

1 Answers
Related