How do I uses Python's multiprocessing.Value method with .map()?

G Schulz 0
Viewed 31

I'm testing some code using multiprocessing to try to understand it, but I'm struggling to get the .Value to work. What am I doing wrong, it says p doesn't exist?

here's my code:

from multiprocessing import Pool, Value
from ctypes import c_int
if __name__ =="__main__":
    p=Value(c_int,1)


def addone(a):
    print(a)

    with p.get_lock():
        print(p.value)
        p.value += 1
if __name__ =="__main__":
    with Pool() as po:
        po.map(addone,range(19))
    print(p.value)

And I get this error:

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "C:\Users\(I removed this)\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "C:\Users\(I removed this)\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 48, in mapstar
    return list(map(*args))
  File "C:\Users\(I removed this)\Desktop\Python\make\multiTest.py", line 10, in addone
    with p.get_lock():
NameError: name 'p' is not defined
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File (I removed this), line 15, in <module>
    po.map(addone,range(19))
  File "C:\Users\(I removed this)\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "C:\Users\(I removed this)\AppData\Local\Programs\Python\Python39\lib\multiprocessing\pool.py", line 771, in get
    raise self._value
NameError: name 'p' is not defined

What should I do?

multiprocessing undefined python
1 Answers

You only want a single instance of the shared value, p, to be operated on by the pool processes executing your worker function, addone. The way to do this is to use the initalizer and initargs arguments of the multiprocessing.pool.Pool class to initialize a global variable, p, in each pool process with the shared value created by the main process. Passing explicitly p to a multiprocessing pool worker function as an additional argument will not work; this will result in the cryptic error: RuntimeError: Synchronized objects should only be shared between processes through inheritance.

def init_pool_processes(shared_value):
    global p
    p = shared_value

def addone(a):
    print(a)

    with p.get_lock():
        print(p.value)
        p.value += 1

if __name__ =="__main__":
    from multiprocessing import Pool, Value
    from ctypes import c_int

    p = Value(c_int, 1)
    with Pool(initializer=init_pool_processes, initargs=(p,)) as po:
        po.map(addone, range(19))
    print(p.value)
Booboo0
Related