Multiprocessing: long vs short running workers

Viewed 220

I am trying something with multiprocessing with a distributed queue. Is there a reason to choose run over run2? My feeling is run2 is slightly more performant since it does not have the overhead of returning control to the main process.

from multiprocessing import Pool
from foo import my_distributed_queue as queue

def process_msg(msg):
  foo(msg)

def run():
  pool = Pool(4)
  for msg in queue:
    pool.apply_async(process_msg, (msg, ))

def process_msg2():
    for msg in queue:
      foo(msg)

def run2():
  pool = Pool(4)
  for i in range(4):
      pool.apply_async(process_msg2, ())

if __name__ == '__main__'
  run()
  #run2()
0 Answers
Related