Avoid partially initialized module ImportError

user_10180567 0
Viewed 252

Context

Let's say I have the following module:

.
└── my_module
    ├── __init__.py
    ├── my_module_A.py (defines ClassA)
    └── my_module_B.py (defines ClassB, which needs ClassA)

and the files are the following:

# __init__.py

from .my_module_A import ClassA
from .my_module_B import ClassB

# my_module_A.py

from my_module import ClassB

class ClassA:

    def __init__(self, class_b: ClassB):
        pass

# my_module_B.py

class ClassB:
    def __init__(self):
        pass

The important thing is that ClassB requires an instance of ClassA to be instantiated.

Problem

Importing the module my_module raises an ImportError:

>>> # Working directory is the parent of my_module
>>> import my_module
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "foo/Module/my_module/__init__.py", line 3, in <module>
    from .my_module_A import ClassA
  File "foo/Module/my_module/my_module_A.py", line 3, in <module>
    from my_module import ClassB
ImportError: cannot import name 'ClassB' from partially initialized module 'my_module' (most likely due to a circular import) (foo/Module/my_module/__init__.py)

The problem seems to be the following. When importing my_module, Python goes through its __init__.py and starts by importing ClassA from my_module_A. This file imports itself ClassB from my_module_B so this class is imported for my_module as well. After my_module_A, Python tries to import ClassB from my_module ; but it was already imported by my_module_A, so Python raises ImportError.

Python suggests that the error may come from a circular import, but this is not the case here. I really think that the problem is that the class ClassB is imported twice: if I do not import my_module_B in __init__.py and use a relative import for my_module_B in my_module_A, the import is successful and there is no exception raised.

Finding a solution

A possible solution is to use only relative imports in python files in my_module. The problem is that some submodules in my_module required themselves a lot of classes from my_module (about 20 for some of them) and it is ugly and inconvenient for maintainers having to know precise file names where classes are defined. So my question is: how can I keep using from my_modyle import … imports in my_module submodules? Thanks a lot.

P.S. Some of you may suggest to rethink my module organization. I thought of that but there is no canonical way to do it and this flat structure is truly easier and clearer. So if possible I would like to avoid this.

Python version: 3.9.1

python-3.x python-import
1 Answers

The source of the problem is ClassB hasn't yet been loaded into my_module (in __init__.py) when my_module_A is being imported (this is hinted by the "partially initialized" bit in the error message). You'll need to manually order the imports in __init__.py so that dependencies come before the modules that use them:

# __init__.py

from .my_module_B import ClassB
from .my_module_A import ClassA  # uses ClassB

A more robust solution is to skip __init__.py and import directly from my_module_B, but as you say, this gets unwieldy with lots of modules.

# my_module_A.py

# relative imports also work here
from my_module.my_module_B import ClassB

class ClassA:
    def __init__(self, class_b: ClassB):
        pass
user_48548210
Related