How do I run all Python unit tests in a directory?

Question

I have a directory that contains my Python unit tests. Each unit test module is of the form test_*.py. I am attempting to make a file called all_test.py that will, you guessed it, run all files in the aforementioned test form and return the result. I have tried two methods so far; both have failed. I will show the two methods, and I hope someone out there knows how to actually do this correctly.

For my first valiant attempt, I thought "If I just import all my testing modules in the file, and then call this unittest.main() doodad, it will work, right?" Well, turns out I was wrong.

import glob
import unittest

testSuite = unittest.TestSuite()
test_file_strings = glob.glob('test_*.py')
module_strings = [str[0:len(str)-3] for str in test_file_strings]

if __name__ == "__main__":
     unittest.main()

This did not work, the result I got was:

$ python all_test.py 

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK

For my second try, I though, ok, maybe I will try to do this whole testing thing in a more "manual" fashion. So I attempted to do that below:

import glob
import unittest

testSuite = unittest.TestSuite()
test_file_strings = glob.glob('test_*.py')
module_strings = [str[0:len(str)-3] for str in test_file_strings]
[__import__(str) for str in module_strings]
suites = [unittest.TestLoader().loadTestsFromName(str) for str in module_strings]
[testSuite.addTest(suite) for suite in suites]
print testSuite 

result = unittest.TestResult()
testSuite.run(result)
print result

#Ok, at this point I have a result
#How do I display it as the normal unit test command line output?
if __name__ == "__main__":
    unittest.main()

This also did not work, but it seems so close!

$ python all_test.py 
]>]>]>


----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK

I seem to have a suite of some sort, and I can execute the result. I am a little concerned about the fact that it says I have only run=1, seems like that should be run=2, but it is progress. But how do I pass and display the result to main? Or how do I basically get it working so I can just run this file, and in doing so, run all the unit tests in this directory?

user_161289 · Answer

I tried various approaches but all seem flawed or I have to makeup some code, that's annoying. But there's a convinient way under linux, that is simply to find every test through certain pattern and then invoke them one by one.

find . -name 'Test*py' -exec python '{}' \;

and most importantly, it definitely works.

find . -name 'Test*py' -exec python '{}' \;

and most importantly, it definitely works.

user_1287554 · Answer

This is an old question, but what worked for me now (in 2019) is:

python -m unittest *_test.py

All my test files are in the same folder as the source files and they end with _test.

user_9597465 · Answer

I just created a discover.py file in my base test directory and added import statements for anything in my sub directories. Then discover is able to find all my tests in those directories by running it on discover.py

python -m unittest discover ./test -p '*.py'

# /test/discover.py
import unittest

from test.package1.mod1 import XYZTest
from test.package1.package2.mod2 import ABCTest
...

if __name__ == "__main__"
    unittest.main()

user_4379130 · Answer

This BASH script will execute the python unittest test directory from ANYWHERE in the file system, no matter what working directory you are in: its working directory always be where that test directory is located.

ALL TESTS, independent $PWD

unittest Python module is sensitive to your current directory, unless you tell it where (using discover -s option).

This is useful when staying in the ./src or ./example working directory and you need a quick overall unit test:

#!/bin/bash
this_program="$0"
dirname="`dirname $this_program`"
readlink="`readlink -e $dirname`"

python -m unittest discover -s "$readlink"/test -v

SELECTED TESTS, independent $PWD

I name this utility file: runone.py and use it like this:

runone.py

#!/bin/bash
this_program="$0"
dirname="`dirname $this_program`"
readlink="`readlink -e $dirname`"

(cd "$dirname"/test; python -m unittest $1)

No need for a test/__init__.py file to burden your package/memory-overhead during production.

user_18131146 · Answer

Encountered the same issue.

The solution is to add an empty __init__.py to each folder and uses python -m unittest discover -s

Project Structure

tests/
  __init__.py
  domain/
    value_object/
      __init__.py
      test_name.py
    __init__.py
  presentation/
    __init__.py
    test_app.py

And running the command

python -m unittest discover -s tests/domain

To get the expected outcome

.
----------------------------------------------------------------------
Ran 1 test in 0.007s

user_5499743 · Answer

I have no package and as mentioned on this page, this is creating issue while issing dicovery. So, I used the following solution. All the test result will be put in a given output folder.

RunAllUT.py:

"""
The given script is executing all the Unit Test of the project stored at the
path %relativePath2Src% currently fixed coded for the given project. 

Prerequired:
    - Anaconda should be install
    - For the current user, an enviornment called "mtToolsEnv" should exists
    - xmlrunner Library should be installed
"""

import sys
import os
import xmlrunner
from Repository import repository 

relativePath2Src="./../.."
pythonPath=r'"C:\Users\%USERNAME%\.conda\envs\YourConfig\python.exe"' 
outputTestReportFolder=os.path.dirname(os.path.abspath(__file__))+r'	est-reports' #subfolder in current file path

class UTTesting():
    """
    Class tto run all the UT of the project
    """
    def __init__(self):
        """
        Initiate instance

        Returns
        -------
        None.

        """
        self.projectRepository = repository() 
        self.UTfile = [] #List all file
    
    def retrieveAllUT(self):
        """
        Generate the list of UT file in the project

        Returns
        -------
        None.

        """
        print(os.path.realpath(relativePath2Src))
        self.projectRepository.retriveAllFilePaths(relativePath2Src)
        #self.projectRepository.printAllFile() #debug
        for file2scan in self.projectRepository.devfile:
            if file2scan.endswith("_UT.py"):
                self.UTfile.append(file2scan)
                print(self.projectRepository.devfilepath[file2scan]+'/'+file2scan)
                
    
    def runUT(self,UTtoRun):
        """
        Run a single UT

        Parameters
        ----------
        UTtoRun : String
            File Name of the UT

        Returns
        -------
        None.

        """
        print(UTtoRun)
        if UTtoRun in self.projectRepository.devfilepath:
            UTtoRunFolderPath=os.path.realpath(os.path.join(self.projectRepository.devfilepath[UTtoRun]))
            UTtoRunPath = os.path.join(UTtoRunFolderPath, UTtoRun)
        print(UTtoRunPath)
        
        #set the correct execution context & run the test
        os.system(" cd " + UTtoRunFolderPath + \
                  " & " + pythonPath + " " + UTtoRunPath + " " + outputTestReportFolder )
        
        
    def runAllUT(self):
        """
        Run all the UT contained in self
        The function "retrieveAllUT" sjould ahve been performed before

        Returns
        -------
        None.

        """
        for UTfile in self.UTfile:
            self.runUT(UTfile)
                
    
                
if __name__ == "__main__":
    undertest=UTTesting()
    undertest.retrieveAllUT()
    undertest.runAllUT()

In my project specific, I have a class that I used in other script. This might be an overkill for your usecase.

Repository.py

import os

class repository():
    """
    Class that decribed folder and file in a repository 
    """
    def __init__(self):
        """
        Initiate instance

        Returns
        -------
        None.

        """
        self.devfile = [] #List all file
        self.devfilepath = {} #List all file paths

    def retriveAllFilePaths(self,pathrepo):
        """
        Retrive all files and their path in the class

        Parameters
        ----------
        pathrepo : Path used for the parsin

        Returns
        -------
        None.

        """
        for path, subdirs, files in os.walk(pathrepo):
            for file_name in files:
                self.devfile.append(file_name)
                self.devfilepath[file_name] = path
                
    def printAllFile(self):
        """
        Display all file with paths

        Parameters
        ----------
        def printAllFile : TYPE
            DESCRIPTION.

        Returns
        -------
        None.

        """
        for file_loop in self.devfile:
            print(self.devfilepath[file_loop]+'/'+file_loop)

In your test files, you need to have a main like this:

if __name__ == "__main__":
    import xmlrunner
    import sys
    
    if len(sys.argv) > 1:
        outputFolder = sys.argv.pop() #avoid conflic with unittest.main
    else:
        outputFolder = r'test-reports'
    print("Report will be created and store there: " + outputFolder)
    
    unittest.main(testRunner=xmlrunner.XMLTestRunner(output=outputFolder))

How do I run all Python unit tests in a directory?

18 Answers