See Background below for a motivation on why I don't think restructuring the code is a good solution.
Let's imagine I have three modules with the following contents (the structure of the code in the codebase I encountered this problem is much more complicated and moving the code around slightly does not work):
# myapp/a.py
from . import b, c
class A:
def to_b(self): return b.B()
def to_c(self): return c.C()
# myapp/b.py
from . import a
class B(a.A): pass
# myapp/c.py
from . import a
class C(a.A): pass
This does not work. Regardless of which of these modules I try to import, I get an error:
cs ~/test$ python3.6 -c 'import myapp.a'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "myapp/a.py", line 1, in <module>
from . import b, c
File "myapp/b.py", line 3, in <module>
class B(a.A): pass
AttributeError: module 'myapp.a' has no attribute 'A'
The modules are not loaded in the order necessary for this to work. A needs to be fully loaded before B and C can be loaded. If we change myapp/b.py's import statement to from . import c, a this is achieved and it works.
But this feels like I'm tricking the runtime into doing what I want instead of declaring it in the form intended by the language designers. Removing or adding an import in the wrong place can lead to an error that is almost un-diagnosable by someone who does not have a very deep understanding of the issues at play here (mind you, I'm facing this issue not with 3, but with about 20 files and I had to deal with even larger, tangled import graphs in the past).
Another trick is to move the import statement in myapp/a.py to the bottom of the file. This, again, feels like a hack and is equally brittle when imports are added or removed.
I'm aware that the import statement could also be moved into the functions, but this has some runtime cost and requires lambda expressions to be converted to functions.
My question: What is the intended way to enforce the required module load order in a situation like this? Extra points if it does not feel like a hack and happens in a single or few places so that it can be easily documented.
Background
This problem came up when working on a large-ish Django project, specifically when splitting its models.py (2.5 k lines/80 classes) into multiple more manageable files (about 20). My question is unrelated to Django or ORMs in general. But the way Django database models are declared lends itself to this issue.
For those not familiar with, models (≈ database tables) are declared using Python classes, foreign keys using references to the other class objects. See this abbreviated example from the documentation of Dango's ORM:
class Musician(models.Model):
[...]
class Album(models.Model):
artist = models.ForeignKey(Musician)
This means that classes need to be declared in a specific order as references are processed while the class body is run.
The model of the project I'm working on is big and complicated and so we added a lot of helper functions to the model classes, many of which access other models. E.g.:
class Musician(models.Model):
def get_albums(self):
return Album.objects.filter(artist=self)
(The actual queries are usually much bigger, hence we want to abstract them)
This example works without issues because class Album will have been defined when Musician.get_albums() is called.
Now imagine we want to move Musician and Album into separate files. Musician depends on Album and Album depends on Musician, requiring a circular import.
Now I could move get_albums() into its separate, third file, but that would place two related and intertwined pieces of code into separate files and change the API exported by Models. I don't think that's a good solution because the placement of the function was not an issue when all model definitions were in a single file.
This is the basis of the problem I described above, in a simplified situation.