Circular import dependency in Python

Viewed 83648

Let's say I have the following directory structure:

a\
    __init__.py
    b\
        __init__.py
        c\
            __init__.py
            c_file.py
        d\
            __init__.py
            d_file.py

In the a package's __init__.py, the c package is imported. But c_file.py imports a.b.d.

The program fails, saying b doesn't exist when c_file.py tries to import a.b.d. (And it really doesn't exist, because we were in the middle of importing it.)

How can this problem be remedied?

7 Answers

Circular Dependencies due to Type Hints

With type hints, there are more opportunities for creating circular imports. Fortunately, there is a solution using the special constant: typing.TYPE_CHECKING.

The following example defines a Vertex class and an Edge class. An edge is defined by two vertices and a vertex maintains a list of the adjacent edges to which it belongs.

Without Type Hints, No Error

File: vertex.py

class Vertex:
    def __init__(self, label):
        self.label = label
        self.adjacency_list = []

File: edge.py

class Edge:
    def __init__(self, v1, v2):
        self.v1 = v1
        self.v2 = v2

Type Hints Cause ImportError

ImportError: cannot import name 'Edge' from partially initialized module 'edge' (most likely due to a circular import)

File: vertex.py

from typing import List
from edge import Edge


class Vertex:
    def __init__(self, label: str):
        self.label = label
        self.adjacency_list: List[Edge] = []

File: edge.py

from vertex import Vertex


class Edge:
    def __init__(self, v1: Vertex, v2: Vertex):
        self.v1 = v1
        self.v2 = v2

Solution using TYPE_CHECKING

File: vertex.py

from typing import List, TYPE_CHECKING

if TYPE_CHECKING:
    from edge import Edge


class Vertex:
    def __init__(self, label: str):
        self.label = label
        self.adjacency_list: List['Edge'] = []

File: edge.py

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from vertex import Vertex


class Edge:
    def __init__(self, v1: 'Vertex', v2: 'Vertex'):
        self.v1 = v1
        self.v2 = v2

Quoted vs. Unquoted Type Hints

In versions of Python prior to 3.10, conditionally imported types must be enclosed in quotes, making them “forward references”, which hides them from the interpreter runtime.

In Python 3.7, 3.8, and 3.9, a workaround is to use the following special import.

from __future__ import annotations

This enables using unquoted type hints combined with conditional imports.

Python 3.10 (See PEP 563 -- Postponed Evaluation of Annotations)

In Python 3.10, function and variable annotations will no longer be evaluated at definition time. Instead, a string form will be preserved in the respective annotations dictionary. Static type checkers will see no difference in behavior, whereas tools using annotations at runtime will have to perform postponed evaluation.

The string form is obtained from the AST during the compilation step, which means that the string form might not preserve the exact formatting of the source. Note: if an annotation was a string literal already, it will still be wrapped in a string.

I suggest the following pattern. Using it will allow auto-completion and type hinting to work properly.

cyclic_import_a.py

import playground.cyclic_import_b

class A(object):
    def __init__(self):
        pass

    def print_a(self):
        print('a')

if __name__ == '__main__':
    a = A()
    a.print_a()

    b = playground.cyclic_import_b.B(a)
    b.print_b()

cyclic_import_b.py

import playground.cyclic_import_a

class B(object):
    def __init__(self, a):
        self.a: playground.cyclic_import_a.A = a

    def print_b(self):
        print('b1-----------------')
        self.a.print_a()
        print('b2-----------------')

You cannot import classes A & B using this syntax

from playgroud.cyclic_import_a import A
from playground.cyclic_import_b import B

You cannot declare the type of parameter a in class B __ init __ method, but you can "cast" it this way:

def __init__(self, a):
    self.a: playground.cyclic_import_a.A = a
Related