multiprocessing pool hangs in jupyter notebook

user_360826 0
Viewed 2719

I have a very simple script which is the following:

import multiprocessing as multi

def call_other_thing_with_multi():
    P = multi.Pool(3)
    P.map(other_thing, range(0,5))
    P.join()


def other_thing(arg):
    print(arg)
    return arg**2.

call_other_thing_with_multi()

When I call this, my code hangs at perpetuity. This is on windows with python 2.7.

Thanks for any guidance!

python-2.7 multiprocessing
1 Answers

As per documentation, you need to call close() before join():

import multiprocessing as multi

def call_other_thing_with_multi():
    P = multi.Pool(3)
    P.map(other_thing, range(0,5))
    P.close() # <-- calling close before P.join()
    P.join()
    print('END')

def other_thing(arg):
    print(arg)
    return arg**2.

call_other_thing_with_multi()

Prints:

0
1
2
3
4
END

EDIT: Better is use context manager, to not forget to call close():

def call_other_thing_with_multi():
    with multi.Pool(3) as P:
        P.map(other_thing, range(0,5))
    print('END')
user_100359850
Related