Importing modules from parent folder

Viewed 1200586

I am running Python 2.5.

This is my folder tree:

ptdraft/
  nib.py
  simulations/
    life/
      life.py

(I also have __init__.py in each folder, omitted here for readability)

How do I import the nib module from inside the life module? I am hoping it is possible to do without tinkering with sys.path.

Note: The main module being run is in the ptdraft folder.

29 Answers

I posted a similar answer also to the question regarding imports from sibling packages. You can see it here.

Solution without sys.path hacks

Summary

  • Wrap the code into one folder (e.g. packaged_stuff)
  • Create a setup.py script where you use setuptools.setup().
  • Pip install the package in editable state with pip install -e <myproject_folder>
  • Import using from packaged_stuff.modulename import function_name

Setup

I assume the same folder structure as in the question

.
└── ptdraft
    ├── __init__.py
    ├── nib.py
    └── simulations
        ├── __init__.py
        └── life
            ├── __init__.py
            └── life.py

I call the . the root folder, and in my case it is located in C:\tmp\test_imports.

Steps

  1. Add a setup.py to the root folder -- The contents of the setup.py can be simply
    from setuptools import setup, find_packages

    setup(name='myproject', version='1.0', packages=find_packages())

Basically "any" setup.py would work. This is just a minimal working example.

  1. Use a virtual environment

If you are familiar with virtual environments, activate one, and skip to the next step. Usage of virtual environments are not absolutely required, but they will really help you out in the long run (when you have more than 1 project ongoing..). The most basic steps are (run in the root folder)

  • Create virtual env
    • python -m venv venv
  • Activate virtual env
    • . venv/bin/activate (Linux) or ./venv/Scripts/activate (Win)
  • Deactivate virtual env
    • deactivate (Linux)

To learn more about this, just Google out "python virtualenv tutorial" or similar. You probably never need any other commands than creating, activating and deactivating.

Once you have made and activated a virtual environment, your console should give the name of the virtual environment in parenthesis

PS C:\tmp\test_imports> python -m venv venv
PS C:\tmp\test_imports> .\venv\Scripts\activate
(venv) PS C:\tmp\test_imports>
  1. pip install your project in editable state

Install your top level package myproject using pip. The trick is to use the -e flag when doing the install. This way it is installed in an editable state, and all the edits made to the .py files will be automatically included in the installed package.

In the root directory, run

pip install -e . (note the dot, it stands for "current directory")

You can also see that it is installed by using pip freeze

(venv) PS C:\tmp\test_imports> pip install -e .
Obtaining file:///C:/tmp/test_imports
Installing collected packages: myproject
  Running setup.py develop for myproject
Successfully installed myproject
(venv) PS C:\tmp\test_imports> pip freeze
myproject==1.0
  1. Import by prepending mainfolder to every import

In this example, the mainfolder would be ptdraft. This has the advantage that you will not run into name collisions with other module names (from python standard library or 3rd party modules).


Example Usage

nib.py

def function_from_nib():
    print('I am the return value from function_from_nib!')

life.py

from ptdraft.nib import function_from_nib

if __name__ == '__main__':
    function_from_nib()

Running life.py

(venv) PS C:\tmp\test_imports> python .\ptdraft\simulations\life\life.py
I am the return value from function_from_nib!

It seems that the problem is not related to the module being in a parent directory or anything like that.

You need to add the directory that contains ptdraft to PYTHONPATH

You said that import nib worked with you, that probably means that you added ptdraft itself (not its parent) to PYTHONPATH.

In a Jupyter Notebook (opened with Jupyter LAB or Jupyter Notebook)

As long as you're working in a Jupyter Notebook, this short solution might be useful:

%cd ..
import nib

It works even without an __init__.py file.

I tested it with Anaconda3 on Linux and Windows 7.

I found the following way works for importing a package from the script's parent directory. In the example, I would like to import functions in env.py from app.db package.

.
└── my_application
    └── alembic
        └── env.py
    └── app
        ├── __init__.py
        └── db
import os
import sys
currentdir = os.path.dirname(os.path.realpath(__file__))
parentdir = os.path.dirname(currentdir)
sys.path.append(parentdir)

Above mentioned solutions are also fine. Another solution to this problem is

If you want to import anything from top level directory. Then,

from ...module_name import *

Also, if you want to import any module from the parent directory. Then,

from ..module_name import *

Also, if you want to import any module from the parent directory. Then,

from ...module_name.another_module import *

This way you can import any particular method if you want to.

Two line simplest solution

import os, sys
sys.path.insert(0, os.getcwd()) 

If parent is your working directory and you want to call another child modules from child scripts.

You can import all child modules from parent directory in any scripts and execute it as

python child_module1/child_script.py

Though the original author is probably no longer looking for a solution, but for completeness, there one simple solution. It's to run life.py as a module like this:

cd ptdraft
python -m simulations.life.life

This way you can import anything from nib.py as ptdraft directory is in the path.

I think you can try this in that specific example, but in python 3.6.3 enter image description here

In a Linux system, you can create a soft link from the "life" folder to the nib.py file. Then, you can simply import it like:

import nib

I have a solution specifically for git-repositories.

First I used sys.path.append('..') and similar solutions. This causes especially problems if you are importing files which are themselves importing files with sys.path.append('..').

I then decided to always append the root directory of the git repository. In one line it would look like this:

sys.path.append(git.Repo('.', search_parent_directories=True).working_tree_dir)

Or in more details like this:

import os
import sys
import git
def get_main_git_root(path):
    main_repo_root_dir = git.Repo(path, search_parent_directories=True).working_tree_dir
    return main_repo_root_dir
main_repo_root_dir = get_main_git_root('.')
sys.path.append(main_repo_root_dir)

For the original question: Based on what the root directory of the repository is, the import would be

import ptdraft.nib

or

import nib

Our folder structure:

/myproject
  project_using_ptdraft/
    main.py
  ptdraft/
    __init__.py
    nib.py
    simulations/
      __init__.py
      life/
        __init__.py
        life.py

The way I understand this is to have a package-centric view. The package root is ptdraft, since it's the top most level that contains __init__.py

All the files within the package can use absolute paths (that are relative to package root) for imports, for example in life.py, we have simply:

import ptdraft.nib

However, to run life.py for package dev/testing purposes, instead of python life.py, we need to use:

cd /myproject
python -m ptdraft.simulations.life.life

Note that we didn't need to fiddle with any path at all at this point.


Further confusion is when we complete the ptdraft package, and we want to use it in a driver script, which is necessarily outside of the ptdraft package folder, aka project_using_ptdraft/main.py, we would need to fiddle with paths:

import sys
sys.path.append("/myproject")  # folder that contains ptdraft
import ptdraft
import ptdraft.simulations

and use python main.py to run the script without problem.

Helpful links:

I had a problem where I had to import a Flask application, that had an import that also needed to import files in separate folders. This is partially using Remi's answer, but suppose we had a repository that looks like this:

.
└── service
    └── misc
        └── categories.csv
    └── test
        └── app_test.py
    app.py
    pipeline.py

Then before importing the app object from the app.py file, we change the directory one level up, so when we import the app (which imports the pipeline.py), we can also read in miscellaneous files like a csv file.

import os,sys,inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0,parentdir)

os.chdir('../')
from app import app

After having imported the Flask app, you can use os.chdir('./test') so that your working directory is not changed.

It's seems to me that you don't really need to import the parent module. Let's imagine that in nib.py you have func1() and data1, you need to use in life.py

nib.py

import simulations.life.life as life
def func1():
   pass
data1 = {}
life.share(func1, data1)

life.py

func1 = data1 = None

def share(*args):
   global func1, data1
   func1, data1 = args

And now you have the access to func1 and data in life.py. Of course you have to be careful to populate them in life.py before you try to use them,

I made this library to do this.

https://github.com/fx-kirin/add_parent_path

# Just add parent path
add_parent_path(1)

# Append to syspath and delete when the exist of with statement.
with add_parent_path(1):
   # Import modules in the parent path
   pass

This is the simplest solution that works for me:

from ptdraft import nib

Although it is against all rules, I still want to mention this possibility:

You can first copy the file from the parent directory to the child directory. Next import it and subsequently remove the copied file:

for example in life.py:

import os
import shutil

shutil.copy('../nib.py', '.')
import nib
os.remove('nib.py')

# now you can use it just fine:
nib.foo()

Of course there might arise several problems when nibs tries to import/read other files with relative imports/paths.

This works for me to import things from a higher folder.

import os
os.chdir('..')
Related