I'm trying to figure out why, when I attempt to log from a process forked using multiprocessing.Process, the log message doesn't honor the logging configuration set in the constructor, however, when I fork the same process using os.fork(), it does. Following is a pared-down code sample to demonstrate:
from multiprocessing import Process
import logging
import logging.config
import os
import sys
class Test():
def __init__(self):
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
ch.setFormatter(formatter)
self.log = logging.getLogger(__name__)
self.log.setLevel(logging.DEBUG)
self.log.addHandler(ch)
def test_log(self):
self.log.warning(f'{os.getpid()}')
def mp_test(self):
self.log.warning('starting multiprocessing test')
proc = Process(target=self.test_log)
proc.start()
proc.join()
def fork_test(self):
self.log.warning('starting fork test')
if os.fork() == 0:
self.test_log()
else:
os.wait()
if __name__ == '__main__':
test = Test()
test.mp_test()
test.fork_test()
Output:
❯ python3 concurrent.py
2022-09-06 19:52:20,933 - __main__ - WARNING - starting multiprocessing test
68926
2022-09-06 19:52:20,977 - __main__ - WARNING - starting fork test
2022-09-06 19:52:20,978 - __main__ - WARNING - 68927
Why is the configuration for self.log not set anymore when self.test_log() is called by multiprocessing.Process? I was under the impression that multiprocessing.Process and os.fork() basically did the same thing and that the new process gets a copy of the parent's address space. I've probably misunderstood something but I'm really keen to find out what that is. Thanks.