Python multiprocessing Pool Interaction With Namespace At Creation

Viewed 948

We know that multiprocessing.Pool must be initialized after the definitions of functions to run on them. However I found the code below is inscrutable for me

import os
from multiprocessing import Pool

def func(i): print('first')

pool1 = Pool(2)
pool1.map(func, range(2))         #map-1

def func(i): print('second')
func2 = func

print('------')
pool1.map(func,  range(2))        #map-2
pool1.map(func2,  range(2))       #map-3

pool2 = Pool(2)
print('------')
pool2.map(func,   range(2))       #map-4
pool2.map(func2,  range(2))       #map-5

The output (python2.7 and python3.4 on linux) is

first         #map-1
first
------
first         #map-2
first
first         #map-3
first
------
second        #map-4
second
second        #map-5
second

map-2 print 'first' just as we expected. But how does map-3 find the name func2? I mean pool1 is initialized before the func2 's first occurrence. So func2 = func is indeed executed, while def func(i): print('second') is not. Why?

And if I define func2 directly by

def func2(i): print('second')

Then map-3 won't find name func2 as mentioned by many posts, eg. this one. What's the difference between two cases?

As I understand the arguments are passed to the slave processes by pickling, but how does pool pass the called function to other processes? Or how do sub-processes find the called function?

1 Answers
Related