Reuse code for scripts in different folders

user_4698922 0
Viewed 59

Looking for a way to import common.py into read.py and write.py without pasting boiler-plate code in each script, and without any clunky sys.path foolery at the top of each script. I have already tried littering __init__.py everywhere.

Dir structure:

parent/
    common.py
    script1/
        config.json
        read.py
        write.py
        ...
    script2/
        config.json
        read.py
        write.py
        ...
    script3/
        config.json
        read.py
        write.py
        ...
    ...

They should all import code like this from common.py:

import sys
import logging
from pathlib import Path

import uvloop

file_path = Path(__file__).resolve()  # remove /../ in path
project_dir = file_path.parent.parent.parent
data_dir = project_dir.parent / 'data'
# Allow imports from project dir
sys.path.insert(1, str(project_dir))


def init_stuff(_file_, logger_name=None):
    uvloop.install()
    lg = logging.getLogger(logger_name)
    ...
    return stuff

Again, to re-iterate once more, I'm looking for a better solution than having to use sys.path.

python-import python
2 Answers

parent should be in your PYTHONPATH environment variable however you are calling python. e.g. if you are calling from a bash command line export PYTHONPATH="/path/to/parent"

Then in both read.py and write.py you would import whatever you want from common. e.g. from common import Foo

user_38214250

Quoting from a great article by Geir Arne Hjelle

While it’s possible to organize your imports by using the current directory as well as by manipulating PYTHONPATH and even sys.path, the process is often unruly and prone to errors

He suggests to create a virtual environment for your project and install it in editable mode. By doing so you don't have to worry about reinstalling it every time you change your scripts.

Here you have the folder and file structure of your project.

root/
    src/
        package/
            __init__.py
            read.py
            write.py
        __init__.py
        common.py
    pyproject.toml
    setup.cfg

Leave those __init__.py files and pyproject.toml blank. The setup.cfg typically contains details about packaging. You don't need them this time, you simply have to specify a package name (maybe not even that) and where to look for packages (ie folders with an __init__.py file).

# setup.cfg

[metadata]
name = my-package

[options]
package_dir =
    = src
packages = find:
python_requires = >=3.6

[options.packages.find]
where = src

Now, create a virtual environment, activate it, and then install the project in editable mode

...\root> python -m venv venv
...\root> venv\Scripts\activate
(venv)...\root> pip install -e .

For the sake of making an example, say that this is all common.py does

# common.py

print("importing common.py")

common.py is then imported in read.py

# read.py

import common

def read():
    print("reading stuff...")

if __name__ == "__main__":
    read()

You can now run read.py

(venv)...\root> python src\package\read.py
importing common.py
reading stuff...

You can also open a Python console and import your scripts from there

>>> import common
importing common.py
>>> from package.read import read
>>> read()
reading stuff...

Check out poetry as well, it's a package for dependancy management and packaging that gives you practically the same result for free.

user_120407510
Related