How to stop all tests from inside a test or setUp using unittest?

Viewed 21043

I'm extending the python 2.7 unittest framework to do some function testing. One of the things I would like to do is to stop all the tests from running inside of a test, and inside of a setUpClass() method. Sometimes if a test fails, the program is so broken it is no longer of any use to keep testing, so I want to stop the tests from running.

I noticed that a TestResult has a shouldStop attribute, and a stop() method, but I'm not sure how to get access to that inside of a test.

Does anyone have any ideas? Is there a better way?

8 Answers

The OP was about python 2.7. Skip ahead a decade, and for python 3.1 and above, the way to skip tests in python unittest has had an upgrade, but the documentation could use some clarification (IMHO):

The documentation covers the following:

  • Skip All tests after first failure: use failfast (only useful if you really don't want to continue any further tests at all, including in other unrelated TestCase classes)
  • Skip All tests in a TestCase class: decorate class with @unittest.skip(), etc.
  • Skip a single method within a TestCase: decorate method with @unittest.skip(), etc.
  • Conditionally skip a method or a class: decorate with @unittest.skipIf() or @unittest.skipUnless() etc.
  • Conditionally skip a method, but not until something within that method runs: use self.skipTest() inside the method (this will skip that method, and ONLY that method, not subsequent methods)

The documentation does not cover the following (as of this writing):

  1. Skip all tests within a TestCase class if a condition is met inside the setUpClass method: solution from this post raise unittest.SkipTest("skip all tests in this class") (there may be another way, but I'm unaware)
  2. Skip all subsequent test methods in a TestCase class after a condition is met in one of the first tests, but still continue to test other unrelated TestCase classes. For this, I propose the following solution...

This solution assumes that you encounter the "bad state" in the middle of a test method, and which could only be noticed in a test method ONLY (i.e., it is not something that could have been determined in the setUpClass method, for whatever reason). Indeed the setUpClass method is the best location for determining whether to proceed if the initial conditions aren't right, but sometimes (as I've encountered) you just don't know until you run some test method. This solution assumes that test methods are in alphabetical order and that subsequent tests methods that you don't want to run after encountering the "bad" state follow alphabetically.

import unittest

class SkipMethodsConditionally(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        #this class variable maintains whether or not test methods should continue
        cls.should_continue = True
        #this class variable represents the state of your system. Replace with function of your own
        cls.some_bad_condition = False

    def setUp(self) -> None:
        """setUp runs before every single test method in this class"""
        if not self.__class__.should_continue:
            self.skipTest("no reason to go on.")

    def test_1_fail(self):
        #Do some work here. Let's assume you encounter a "bad state,"" that could 
        #only be noticed in this first test method only, (i.e., it's not something that
        #can be placed in the setUpClass method, for whatever reason)
        self.__class__.some_bad_condition = True

        if self.__class__.some_bad_condition:
            self.__class__.should_continue = False

        self.assertTrue(False,"this test should fail, rendering the rest of the tests irrelevant")

    def test_2_pass(self):
        self.assertFalse(self.__class__.some_bad_condition,"this test would pass normally if run, but should be skipped, because it would fail")

The above test will yield the following output:

test_1_fail (__main__.SkipMethodsConditionally) ... FAIL
test_2_pass (__main__.SkipMethodsConditionally) ... skipped 'no reason to go on.'
----------------------------------------------------------------------
Ran 2 tests in 0.001s

FAILED (failures=1, skipped=1)
Related