I have two classes A and B.
A should import B dynamically (so I can change B during runtime).
I also want to do some multiprocessing in B.
This is my directory structure:
src\
__init__.py
a\
__init__.py
a.py
b\
__init__.py
b.py
This is a.py:
import os, importlib.util
if __name__ == '__main__':
module_path = 'src\\b\\b.py'
module_name = 'b'
class_name = 'B'
# dynamically importing the module
spec = importlib.util.spec_from_file_location(module_name, module_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
# dynamically instantiating the class
b_class = getattr(module, class_name)
b = b_class()
# start multiprocessing
b.run()
This is b.py:
import multiprocessing as mp
class B():
def f(self):
print('success')
def run(self):
print('run')
p = mp.Process(target=self.f)
p.start()
p.join()
if __name__ == '__main__':
b = B()
b.run()
Running b.py works fine:
> python src\b\b.py
run
success
Running a.py raises an exception:
> python src\a\a.py
run
Traceback (most recent call last):
File "src\a\a.py", line 18, in <module>
b.run()
File "src\b\b.py", line 10, in run
p.start()
File "C:\Program Files\Python38\lib\multiprocessing\process.py", line 121, in start
self._popen = self._Popen(self)
File "C:\Program Files\Python38\lib\multiprocessing\context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:\Program Files\Python38\lib\multiprocessing\context.py", line 327, in _Popen
return Popen(process_obj)
File "C:\Program Files\Python38\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
reduction.dump(process_obj, to_child)
File "C:\Program Files\Python38\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class 'b.B'>: import of module 'b' failed
C:\Users\micha\Projects\Python\PickleQuestion>Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Program Files\Python38\lib\multiprocessing\spawn.py", line 102, in spawn_main
source_process = _winapi.OpenProcess(
OSError: [WinError 87] Falscher Parameter
Why? Is there another way to import B, while keeping the flexibility of not hardcoding it (from b import B)? I want b.py to be in any package, even outside of src.
Python version: 3.8.6