Python unittest: how to run only part of a test file?

The default unittest.main() uses the default test loader to make a TestSuite out of the module in which main is running.

You don't have to use this default behavior.

You can, for example, make three unittest.TestSuite instances.

1. The "fast" subset.

   fast = TestSuite()
   fast.addTests(TestFastThis)
   fast.addTests(TestFastThat)
   

2. The "slow" subset.

   slow = TestSuite()
   slow.addTests(TestSlowAnother)
   slow.addTests(TestSlowSomeMore)
   

3. The "whole" set.

   alltests = unittest.TestSuite([fast, slow])
   

Note that I've adjusted the TestCase names to indicate Fast vs. Slow. You can subclass unittest.TestLoader to parse the names of classes and create multiple loaders.

Then your main program can parse command-line arguments with optparse or argparse (available since 2.7 or 3.2) to pick which suite you want to run, fast, slow or all.

Or, you can trust that sys.argv[1] is one of three values and use something as simple as this

if __name__ == "__main__":
    suite = eval(sys.argv[1])  # Be careful with this line!
    unittest.TextTestRunner().run(suite)

You have basically two ways to do it:

1. Define your own suite of tests for the class
2. Create mock classes of the cluster connection that will return actual data.

I am a strong proponent of he second approach; a unit test should test only a very unit of code, and not complex systems (like databases or clusters). But I understand that it is not always possible; sometimes, creating mock ups is simply too expensive, or the goal of the test is really in the complex system.

Back to option (1), you can proceed in this way:

suite = unittest.TestSuite()
suite.addTest(MyUnitTestClass('quickRunningTest'))
suite.addTest(MyUnitTestClass('otherTest'))

and then passing the suite to the test runner:

unittest.TextTestRunner().run(suite)

More information on the python documentation: http://docs.python.org/library/unittest.html#testsuite-objects

I found another solution, based on how the unittest.skip decorator works. By setting the __unittest_skip__ and __unittest_skip_why__.

Label-based

I wanted to apply a labeling system, to label some tests as quick, slow, glacier, memoryhog, cpuhog, core, and so on.

Then run all 'quick' tests, or run everything except 'memoryhog' tests, your basic whitelist / blacklist setup

Implementation

I implemented this in two parts:

1. First add labels to tests (via a custom @testlabel class decorator)
2. Custom unittest.TestRunner to identify which tests to skip, and modify the testlist content before executing.

Working implementation is in this gist: https://gist.github.com/fragmuffin/a245f59bdcd457936c3b51aa2ebb3f6c

(A fully working example was too long to put here.)

The result being...

$ ./runtests.py --blacklist foo
test_foo (test_things.MyTest2) ... ok
test_bar (test_things.MyTest3) ... ok
test_one (test_things.MyTests1) ... skipped 'label exclusion'
test_two (test_things.MyTests1) ... skipped 'label exclusion'

----------------------------------------------------------------------
Ran 4 tests in 0.000s

OK (skipped=2)

All MyTests1 class tests are skipped, because it has the foo label.

--whitelist also works

I created a decorator that allows for marking tests as slow tests and to skip them using an environment variable

from unittest import skip
import os

def slow_test(func):
    return skipIf('SKIP_SLOW_TESTS' in os.environ, 'Skipping slow test')(func)

Now you can mark your tests as slow like this:

@slow_test
def test_my_funky_thing():
    perform_test()

And skip slow tests by setting the SKIP_SLOW_TESTS environment variable:

SKIP_SLOW_TESTS=1 python -m unittest

Look into using a dedicated testrunner, like py.test, Nose or possibly even zope.testing. They all have command-line options for selecting tests.

Look for example at Nose.

I found this answer trying to figure out how to just run specific classes of tests; for example,

class TestCase1(unittest.TestCase):
    def some_test(self):
        self.assertEqual(True, True)

class TestCase2(unittest.TestCase):
    def some_other_test(self):
        self.assertEqual(False, False)

I wanted a quick way to comment out TestCase1 or TestCase2 that didn't involve me sweep-selecting 100+ lines of code, and I eventually landed on this:

if __name__ == "__main__":
    tests = []
    tests.append("TestCase1")
    # tests.append("TestCase2")
    unittest.main(defaultTest=tests)

It just uses unittest.main()'s defaultTest argument to specify which test classes to run.

Sometimes I run each of my test functions manually. Say my test class looks like this...

class TestStuff(unittest.TestCase):
    def test1():
    def test2():

Then I run this...

t = TestStuff()
t.test1()
t.test2()

(I use the Spyder IDE for data analysis, this might not be ideal for IDEs with a slicker testing tools)