PyInstaller bundle causes FileNotFoundError with multiprocessing spawn method

user_8350610 0
Viewed 76

I have a python application which is bundled using pyinstaller --onefile method. When running with multiprocessing start method spwan, it causes error in middle of the application.

Traceback (most recent call last):
  File "web.py", line 1028, in <module>
  File "PyInstaller/hooks/rthooks/pyi_rth_multiprocessing.py", line 49, in _freeze_support
  File "multiprocessing/spawn.py", line 105, in spawn_main
  File "multiprocessing/spawn.py", line 114, in _main
  File "multiprocessing/spawn.py", line 225, in prepare
  File "multiprocessing/spawn.py", line 277, in _fixup_main_from_path
  File "runpy.py", line 261, in run_path
  File "runpy.py", line 231, in _get_code_from_file
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/_MEIRtGMqX/web.py'
[11797] Failed to execute script 'web' due to unhandled exception!

start method fork seems to be fine, but sometimes it hangs due to resource lock issue, so I prefer spawn. Does any one have any idea why this error occurs?

I have been using freeze_support() as mentioned in multiprocessing documentation even though it doesn't have any impact on Linux.

if __name__ == "__main__":
    freeze_support()
    set_start_method('spawn')

OS - Amazon linux 2

Python - 3.6.8

PyInstaller - 4.10

As I understood about spwan, It runs a new Python interpreter whenever creates a new process and tell it to import the main module and then execute. So my main module is web.py. So the error happens when a new process is created. But this issue is not consistent.

amazon-linux-2 pyinstaller python-3.x multiprocessing python
2 Answers

I think the reason for this problem is that the program cannot find web.py after packaging.
You can try adding the following code where you are using about path.

if getattr(sys, 'frozen', False):
    bundle_dir = sys._MEIPASS
else:
    bundle_dir = os.path.dirname(os.path.abspath(__file__))

this code can help program find the file after packaging.
pyinstaller's document about this:https://pyinstaller.org/en/stable/runtime-information.html

user_191968400

At this point of time, Only one solution I have found is start the multiprocessing with method forkserver. I am not sure whether it leads to any hang in application.

set_start_method('forkserver')

user_83506100
Related