Changing global variable in main process within a spawned process using multiprocessing Pool

user_12102460 0
Viewed 456

My sample code is as follows:

from multiprocessing import Pool
GLOBAL_VAR = 0

def try_change_global_var(notused):
    global GLOBAL_VAR
    print(f"My own GLOBAL_VAR in current process starts as: {GLOBAL_VAR}")
    GLOBAL_VAR = 1
    print("I tried to change GLOBAL_VAR to 1")
    return "change"


if __name__ == '__main__':
    p1 = Pool(4)
    result = p1.map(try_change_global_var, range(4))
    print(result)
    print(f"GLOBAL_VAR in main process is still: {GLOBAL_VAR}")
    p1.close()

I read that every spawned Python process gets its own global variable, but if you take a look at my console output, it seems as if all 4 spawned processes were accessing the same variable, because it was immediately changed from 0 to 1 and it stayed 1. When the 4 processes finished the global variable from main was of course still 0.

My own GLOBAL_VAR in current process starts as: 0
I tried to change GLOBAL_VAR to 1
My own GLOBAL_VAR in current process starts as: 1
I tried to change GLOBAL_VAR to 1
My own GLOBAL_VAR in current process starts as: 1
I tried to change GLOBAL_VAR to 1
My own GLOBAL_VAR in current process starts as: 1
I tried to change GLOBAL_VAR to 1
['change', 'change', 'change', 'change']
GLOBAL_VAR in main process is still: 0

I could change it within main, based on the list of returns from the called function, but that is not what I want. I want to share memory and change the global variable in the main process, from the spawned processes.

I was unable to get a working toy example with multiprocessing.Value and would appreciate some help. The examples I found were not using Pool and were able to pass multiple arguments to the functions they were running in parallel processes.

OS: Windows 10 Python 3.9.5

python-multiprocessing python
1 Answers

This is working exactly as intended. When you spawn a process, you are making a copy of your current environment, and giving it to the new process. Changes made by that process happen in that process, and only in that process.

So every process changes its own GLOBAL_VAR to 1, but that value is completely independent of the GLOBAL_VAR in the main process. Python provides "shared variables" if you need to use a variable across multiple threads, but they are somewhat expensive. (multiprocessing.Value)

user_64574070
Related