Circular dependency - when does it terminate?

user_9482638 0
Viewed 102

I am having trouble understanding how python manages imports.

Let's say I have the following application structure:

application/
- application.py
- model/
-- __init__.py
-- user.py

Let's say that the application.py file imports the model module after creating the db like so:

db = SQLAlchemy(application) 
import model

Let's also say the the model module imports the user.py file like so:

import user

Finally, let's say that the user.py file imports the db instance from the application.py file like so:

from application import db

This seems like a circular dependency to me as the application.py file indirectly requires the user.py file but the user.py file requires the db instance from the application.py file.

I know that this code does work as I tested it but can someone explain exactly how Python handles this, and when it terminates the cyclical cycle.

To summarize the problem, when user.py file imports db from the application.py file it seems to me like it would also call the import model module which creates an infinite loop.

import python-3.x python
1 Answers

Circular imports themselves are not necessarily a problem. Only circular dependencies are. You can see how the circular import is resolved with some simple experimentation:

In mymod1.py

import sys
print("1. from mymod1:", [x for x in sys.modules if x.startswith("mymod")])
import mymod2
print("2. from mymod1:", [x for x in sys.modules if x.startswith("mymod")])

In mymod2.py

import sys
print("1. from mymod2:", [x for x in sys.modules if x.startswith("mymod")])
import mymod3
print("2. from mymod2:", [x for x in sys.modules if x.startswith("mymod")])

In mymod3.py

import sys
print("1. from mymod3:", [x for x in sys.modules if x.startswith("mymod")])
import mymod1
print("2. from mymod3:", [x for x in sys.modules if x.startswith("mymod")])

Now we have a cycle mymod1 -> mymod2 -> mymod3 -> mymod1. Enter the REPL and watch what happens:

>>> import mymod1
1. from mymod1: ['mymod1']  # before mymod1 imported mymod2. note mymod1 is already there!
1. from mymod2: ['mymod1', 'mymod2']  # in mymod2 now, before importing mymod3
1. from mymod3: ['mymod1', 'mymod2', 'mymod3']  # before mymod3 imports mymod1
2. from mymod3: ['mymod1', 'mymod2', 'mymod3']  # mymod3 exit
2. from mymod2: ['mymod1', 'mymod2', 'mymod3']  # mymod2 exit
2. from mymod1: ['mymod1', 'mymod3', 'mymod2']  # mymod1 exit

The key insight here is that the module instance itself is already present in sys.modules before it has completed execution. That means it can be imported again and it will return the existing object, without needing to execute all the module-level code again.

It's natural for components within submodules of a package to have interdependencies. Issues mostly arise when module-scoped code starts actually doing things like trying to connect to a db, so try to avoid scripting anything directly in the module level.

The cause of circular import errors are when something in the module namespace is needed before the module has finished initializing. In this case, the module itself exists but the name you're trying to access might not be there yet.

# mymod.py
import sys
print("var" in vars(sys.modules["mymod"]))
var = "I'm a name in mymod namespace"
print("var" in vars(sys.modules["mymod"]))

Importing mymod would print False and then True, the module's namespace itself is still being mutated as the import is in the process of executing.

An astute reader may have noticed that mymod2 and mymod3 switched places in the output:

2. from mymod2: ['mymod1', 'mymod2', 'mymod3']
2. from mymod1: ['mymod1', 'mymod3', 'mymod2']
                           |_____ wtf? _____|

This is actually no accident! The import machinery, as a final step in loading an module, pulls the actual module out of sys.modules. If you check again within the REPL, mymod1 will now be last.

>>> import mymod1
...
>>> [x for x in sys.modules if x.startswith("mymod")]
['mymod3', 'mymod2', 'mymod1']

I won't describe why the import system does that, because it's not really relevant to the question, but users interested to know the reasons should see this mailing list post from Guido.

user_6740390
Related