Running unittest with typical test directory structure

Viewed 343224

The very common directory structure for even a simple Python module seems to be to separate the unit tests into their own test directory:

new_project/
    antigravity/
        antigravity.py
    test/
        test_antigravity.py
    setup.py
    etc.

My question is simply What's the usual way of actually running the tests? I suspect this is obvious to everyone except me, but you can't just run python test_antigravity.py from the test directory as its import antigravity will fail as the module is not on the path.

I know I could modify PYTHONPATH and other search path related tricks, but I can't believe that's the simplest way - it's fine if you're the developer but not realistic to expect your users to use if they just want to check the tests are passing.

The other alternative is just to copy the test file into the other directory, but it seems a bit dumb and misses the point of having them in a separate directory to start with.

So, if you had just downloaded the source to my new project how would you run the unit tests? I'd prefer an answer that would let me say to my users: "To run the unit tests do X."

23 Answers

The simplest solution for your users is to provide an executable script (runtests.py or some such) which bootstraps the necessary test environment, including, if needed, adding your root project directory to sys.path temporarily. This doesn't require users to set environment variables, something like this works fine in a bootstrap script:

import sys, os

sys.path.insert(0, os.path.dirname(__file__))

Then your instructions to your users can be as simple as "python runtests.py".

Of course, if the path you need really is os.path.dirname(__file__), then you don't need to add it to sys.path at all; Python always puts the directory of the currently running script at the beginning of sys.path, so depending on your directory structure, just locating your runtests.py at the right place might be all that's needed.

Also, the unittest module in Python 2.7+ (which is backported as unittest2 for Python 2.6 and earlier) now has test discovery built-in, so nose is no longer necessary if you want automated test discovery: your user instructions can be as simple as python -m unittest discover.

I've had the same problem for a long time. What I recently chose is the following directory structure:

project_path
├── Makefile
├── src
│   ├── script_1.py
│   ├── script_2.py
│   └── script_3.py
└── tests
    ├── __init__.py
    ├── test_script_1.py
    ├── test_script_2.py
    └── test_script_3.py

and in the __init__.py script of the test folder, I write the following:

import os
import sys
PROJECT_PATH = os.getcwd()
SOURCE_PATH = os.path.join(
    PROJECT_PATH,"src"
)
sys.path.append(SOURCE_PATH)

Super important for sharing the project is the Makefile, because it enforces running the scripts properly. Here is the command that I put in the Makefile:

run_tests:
    python -m unittest discover .

The Makefile is important not just because of the command it runs but also because of where it runs it from. If you would cd in tests and do python -m unittest discover ., it wouldn't work because the init script in unit_tests calls os.getcwd(), which would then point to the incorrect absolute path (that would be appended to sys.path and you would be missing your source folder). The scripts would run since discover finds all the tests, but they wouldn't run properly. So the Makefile is there to avoid having to remember this issue.

I really like this approach because I don't have to touch my src folder, my unit tests or my environment variables and everything runs smoothly.

From the article you linked to:

Create a test_modulename.py file and put your unittest tests in it. Since the test modules are in a separate directory from your code, you may need to add your module’s parent directory to your PYTHONPATH in order to run them:

$ cd /path/to/googlemaps

$ export PYTHONPATH=$PYTHONPATH:/path/to/googlemaps/googlemaps

$ python test/test_googlemaps.py

Finally, there is one more popular unit testing framework for Python (it’s that important!), nose. nose helps simplify and extend the builtin unittest framework (it can, for example, automagically find your test code and setup your PYTHONPATH for you), but it is not included with the standard Python distribution.

Perhaps you should look at nose as it suggests?

I had the same problem, with a separate unit tests folder. From the mentioned suggestions I add the absolute source path to sys.path.

The benefit of the following solution is, that one can run the file test/test_yourmodule.py without changing at first into the test-directory:

import sys, os
testdir = os.path.dirname(__file__)
srcdir = '../antigravity'
sys.path.insert(0, os.path.abspath(os.path.join(testdir, srcdir)))

import antigravity
import unittest

if you run "python setup.py develop" then the package will be in the path. But you may not want to do that because you could infect your system python installation, which is why tools like virtualenv and buildout exist.

Use setup.py develop to make your working directory be part of the installed Python environment, then run the tests.

Python 3+

Adding to @Pierre

Using unittest directory structure like this:

new_project
├── antigravity
│   ├── __init__.py         # make it a package
│   └── antigravity.py
└── test
    ├── __init__.py         # also make test a package
    └── test_antigravity.py

To run the test module test_antigravity.py:

$ cd new_project
$ python -m unittest test.test_antigravity

Or a single TestCase

$ python -m unittest test.test_antigravity.GravityTestCase

Mandatory don't forget the __init__.py even if empty otherwise will not work.

You can't import from the parent directory without some voodoo. Here's yet another way that works with at least Python 3.6.

First, have a file test/context.py with the following content:

import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))

Then have the following import in the file test/test_antigravity.py:

import unittest
try:
    import context
except ModuleNotFoundError:
    import test.context    
import antigravity

Note that the reason for this try-except clause is that

  • import test.context fails when run with "python test_antigravity.py" and
  • import context fails when run with "python -m unittest" from the new_project directory.

With this trickery they both work.

Now you can run all the test files within test directory with:

$ pwd
/projects/new_project
$ python -m unittest

or run an individual test file with:

$ cd test
$ python test_antigravity

Ok, it's not much prettier than having the content of context.py within test_antigravity.py, but maybe a little. Suggestions are welcome.

If you have multiple directories in your test directory, then you have to add to each directory an __init__.py file.

/home/johndoe/snakeoil
└── test
    ├── __init__.py        
    └── frontend
        └── __init__.py
        └── test_foo.py
    └── backend
        └── __init__.py
        └── test_bar.py

Then to run every test at once, run:

python -m unittest discover -s /home/johndoe/snakeoil/test -t /home/johndoe/snakeoil

Source: python -m unittest -h

  -s START, --start-directory START
                        Directory to start discovery ('.' default)
  -t TOP, --top-level-directory TOP
                        Top level directory of project (defaults to start
                        directory)

This BASH script will execute the python unittest test directory from anywhere in the file system, no matter what working directory you are in.

This is useful when staying in the ./src or ./example working directory and you need a quick unit test:

#!/bin/bash

this_program="$0"
dirname="`dirname $this_program`"
readlink="`readlink -e $dirname`"

python -m unittest discover -s "$readlink"/test -v

No need for a test/__init__.py file to burden your package/memory-overhead during production.

This way will let you run the test scripts from wherever you want without messing around with system variables from the command line.

This adds the main project folder to the python path, with the location found relative to the script itself, not relative to the current working directory.

import sys, os

sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.realpath(__file__))))

Add that to the top of all your test scripts. That will add the main project folder to the system path, so any module imports that work from there will now work. And it doesn't matter where you run the tests from.

You can obviously change the project_path_hack file to match your main project folder location.

A simple solution for *nix based systems (macOS, Linux); and probably also Git bash on Windows.

PYTHONPATH=$PWD python test/test_antigravity.py

print statement easily works, unlike pytest test/test_antigravity.py. A perfect way for "scripts", but not really for unittesting.

Of course, I want to do a proper automated testing, I would consider pytest with appropriate settings.

If you are looking for a command line-only solution:

Based on the following directory structure (generalized with a dedicated source directory):

new_project/
    src/
        antigravity.py
    test/
        test_antigravity.py

Windows: (in new_project)

$ set PYTHONPATH=%PYTHONPATH%;%cd%\src
$ python -m unittest discover -s test

See this question if you want to use this in a batch for-loop.

Linux: (in new_project)

$ export PYTHONPATH=$PYTHONPATH:$(pwd)/src  [I think - please edit this answer if you are a Linux user and you know this]
$ python -m unittest discover -s test

With this approach, it is also possible to add more directories to the PYTHONPATH if necessary.

unittest in your project have setup.py file. try:

python3 setup.py build

and

python3 setup.py develop --user

do the work of config paths an so on. try it!

Related