`multiprocessing.Process` are modifying non-shared variables they should not have access to

user_10011266 0
Viewed 92

Processes are mutating things they should not be able to mutate.

A Workerhas a single state variable (an mp.Value). This value is set to -1, and it (the Worker) changes it to 1 in a loop.

However, it seems to be possible to reset that value back to -1 by spawning a second Worker, even though this shares nothing with the original pair. This seems like it should be impossible.

Behavior:

When the second Worker spins up, the state of the first worker (self.state.value ) gets reset to -1. This gets caught, and we print out that an error was discovered.

Code:

import multiprocessing as mp
import time


class Worker:
    def __init__(self, tag, service_state) -> None:
        self.tag = tag
        self.local_state = int(service_state.value)
        self.state = service_state
        self.run_work_loop()

    def run_work_loop(self) -> None:
        print(f"[{self.tag}] Running... {self.state.value} {self.local_state}")
        while True:
            if self.state.value != self.local_state:
                print(f"[{self.tag}] Illegal change. Shared state: {self.state.value} Local State: {self.local_state}")
                break

            elif self.state.value == -1:
                self.state.value = self.local_state = 1
                print(f"[{self.tag}] Set Shared State: {self.state.value} Local State: {self.local_state}.")


if __name__ == "__main__":
    mp.Process(target=Worker, args=("A", mp.Value('i', -1))).start()
    time.sleep(.03)
    mp.Process(target=Worker, args=("B", mp.Value('i', -1))).start()

Output:

[A] Running... -1 -1
[A] Set Shared State: 1 Local State: 1.
[A] Illegal change. Shared state: -1 Local State: 1
[B] Running... -1 -1
[B] Set Shared State: 1 Local State: 1.
python-multiprocessing multiprocessing python
1 Answers

The issue is that you are creating Value instances that immediately go out of scope in the parent process, which makes them get garbage collected. Because of the way Python allocates memory for multiprocessing.Value objects, the second Value ends up using the exact same shared memory location as the first Value, which means the second ends up stomping on the first. You can do some experiments to see this in action. For example, this does not print the warning:

if __name__ == "__main__":
    mp.Process(target=Worker, args=("A", mp.Value('i', -1))).start()
    time.sleep(.03)
    a = mp.Value('i', 1)
    mp.Process(target=Worker, args=("B", mp.Value('i', -1))).start()

The Value we assign to a is initialized to 1, which overwrites the anonymous Value we passed to process "A". Because we overwrite it with 1, no illegal state message is printed. if we instead initialize it to any other value, you will see the warning again. This prints an illegal state message about -2, , for example:

if __name__ == "__main__":
    mp.Process(target=Worker, args=("A", mp.Value('i', -1))).start()
    time.sleep(.03)
    a = mp.Value('i', -2)
    mp.Process(target=Worker, args=("B", mp.Value('i', -1))).start()

Your code should really save the Value instances you create as local variables in your parent process, both to avoid this issue, and because it's pointless to create shared values that you don't actually share. Like this:

if __name__ == "__main__":
    a = mp.Value('i', -1)
    mp.Process(target=Worker, args=("A", a)).start()
    time.sleep(.03)
    b = mp.Value('i', -1)
    mp.Process(target=Worker, args=("B", b)).start()
user_20735950
Related