An attempt has been made to start a new process before the current process has finished its bootstrapping phase in python

user_3446201 0
Viewed 2223

I try to run this code on python

import multiprocessing

manager = multiprocessing.Manager()
final_list = manager.list()

input_list_one = ['one', 'two', 'three', 'four', 'five']
input_list_two = ['six', 'seven', 'eight', 'nine', 'ten']

def worker(data):
    for item in data:
        final_list.append(item)

if __name__ == '__main__':    

    process1 = multiprocessing.Process(target=worker, args=[input_list_one])
    process2 = multiprocessing.Process(target=worker, args=[input_list_two])

    process1.start()
    process2.start()
    process1.join()
    process2.join()

    print(final_list)

But this error is happened: RuntimeError:

An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:

    if __name__ == '__main__':
        freeze_support()
        ...

The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
multiprocessing python
2 Answers

The problems is with the statement:

manager = multiprocessing.Manager()

which does its "magic" by starting a server process. Therefore, this statement needs to be moved to within the if __name__ = '__main__': block along with the creation of the managed list, which now needs to be passed as an additional argument to your process function, worker. In fact, you might as well move all declarations at global scope that do not really need to be there within the if __name__ = '__main__': block for efficiency because they would otherwise be needlessly executed by each new process created.

import multiprocessing

def worker(final_list, data):
    for item in data:
        final_list.append(item)

if __name__ == '__main__':
    manager = multiprocessing.Manager()
    final_list = manager.list()

    input_list_one = ['one', 'two', 'three', 'four', 'five']
    input_list_two = ['six', 'seven', 'eight', 'nine', 'ten']

    process1 = multiprocessing.Process(target=worker, args=[final_list, input_list_one])
    process2 = multiprocessing.Process(target=worker, args=[final_list, input_list_two])

    process1.start()
    process2.start()
    process1.join()
    process2.join()

    print(final_list)

Prints:

['six', 'seven', 'eight', 'nine', 'one', 'ten', 'two', 'three', 'four', 'five']

Let me elaborate a bit on my answer:

You are clearly running on a platform that uses the spawn method to create new processes. This means that to launch a new process a new, empty address space is created and a new instance of the Python interpreter is run against the source. Before the target of your Process instance is invoked, any statement in the source file that is at global scope will first be executed to initialize the process, except any statement within a if __name__ = '__main__': block (because the __name__ value will not be '__main__' for the new process). This is why you need to put code that creates new processes within such a block, i.e. to avoid getting into what would be a recursive loop creating new processes ad infinitum if this went undetected. In your case it did not go undetected and you got the error message you saw.

But even if creating a Manager instance had not resulted in a new process getting created, your program would not have been correct. Having the statement final_list = manager.list() at global scope meant that all three processes in your running program would have been accessing three different instances of final_list.

user_28237190

It happens in windows spawn.py file. As it is Runtime error, we can ignore it when starting the process, using "if name == 'main':" EG: yolo v5 with python 3.8.5

if __name__ == '__main__':
    from yolov5 import train
    train.run()
user_128793250
Related