How can I deal with importing multiple projects that use absolute imports and have the same folder and filenames?
I'm working on ML code where many authors have written code that is not meant to be a pip package, and the folders and filenames have high overlap and imports are absolute assuming the root of their code is the root of PYTHONPATH. A setup is this:
myproject
myversion
mymodel.py
third_party
version1
model.py
util.py
version2
model.py
util.py
With contents like:
# myproject/third_party/version1/model.py
import util
def process(x):
return 2*util.preprocess(x)
# myproject/third_party/version2/model.py
import util
def process(x):
return 3*util.preprocess(x)
# myproject/third_party/version1/util.py
def preprocess(x):
return 1
# myproject/third_party/version2/util.py
def preprocess(x):
return 0
This is what I want to do for example, running PYTHONPATH=/path/to/myproject myversion/mymodel.py
# myproject/myversion/mymodel.py
from third_party.version1.model import process as process1
from third_party.version2.model import process as process2
def myprocess(x):
return x
def compare(x):
return process1(x), process2(x), myprocess(x)
But I'm trapped because each version1 and version2 needs a different path to be able to find their own util!
What is the real solution here? I know if there were relative imports used in version1 and version2 files (not authored by me) it would be easy. I also know if I only used one of the two, it would be easy to just give PYTHONPATH=/path/to/myproject:/path/to/myproject/third_party/version1 and everything would be fine. But I would like to use both. In an ideal world I wouldn't change the code inside version1 and version2 very much or at all.
What I wish I could do is fully encapsulate version1 and version2 code and when run they magically know where to look for their own files. I tried using setuptools but I still have path problems. I suspect bazel can be used for this somehow?
p.s. Using python 3
p.p.s. I uploaded a demo on gh here