How to avoid brittle tests in a function that has configuration options?

user_7225482 0
Viewed 56

Problem Statement

I am trying to build unit tests for a function that performs a series of arithmetic operations on the input it receives. The operations are configured outside of the function (in a package constant in this case).

My issue is that I can write tests that pass when they should but either the tests will break if the external configuration changes or the test will be mostly a repetition of the code in the function to be tested.

The two ways I can think of writing the tests are:

  1. A test that "assumes" a specific configuration. The problem is that the test is fragile and will stop working if I change the configuration. (See test_example1 below)

    • Create a expected result assuming some operations and applying to the input
    • Call the function to test
    • Compare expected result with actual returned by the function to test

  2. I can build a test that uses the configuration to calculate an expected result. The problems are, first, the test depends on a configuration constant outside of the function, and second, the code for the test is the very similar to the tested code, which feels wrong. (See test_example2 below)

    • Create a expected result processing the input according to the configuration constant
    • Call the function to test
    • Compare expected result with actual returned by the function to test

Can you help me figure out what is the right way to unit test a function that performs operations configured outside of it?

Sample Code

import unittest
import operator

OPS = {'+':operator.add,
       '-':operator.sub}

DIRECTIVES = {'calculated_field':[('+','field1'),
                                  ('+','field2')]}

def example(input_dict):
    output_dict = {}
    
    for item,calcs in DIRECTIVES.items():
        output_dict[item] = 0
        for operation, field in calcs:
            output_dict[item] = OPS[operation](output_dict[item],input_dict[field])
        
    return output_dict    

class TestExample(unittest.TestCase):

    item1  = 'field1'
    value1 = 10
    item2  = 'field2'
    value2 = 20
    item3  = 'field3'
    value3 = 5    

    def setUp(self):
        self.input_dict = {self.item1:self.value1,
                           self.item2:self.value2,
                           self.item3:self.value3}
    
    def test_example_option1(self):
        expected_result = {'calculated_field':self.value1+self.value2}
        
        actual_result = example(self.input_dict)
        
        self.assertDictEqual(expected_result,actual_result)
        
    def test_example_option2(self):
        expected_result = {}

        for item,calcs in DIRECTIVES.items():
            expected_result[item] = 0
            for operation, field in calcs:
                expected_result[item] = OPS[operation](expected_result[item],self.input_dict[field])
                
        actual_result = example(self.input_dict)
        
        self.assertDictEqual(expected_result,actual_result)
python-unittest unit-testing python
1 Answers

This is a bit a matter of opinion, but rather than rely strictly on global variables, I might make your function (example(), in this case, I'm assuming) allow passing in optional overrides to the external data it relies on.

For example

def example(ops=OPS, directives=DIRECTIVES):
    ...

This has two advantages: For the sake of testing you can pass in some dummy values for this data (which, presumably, would be smaller and simpler than the real data) and then test for a known-correct output given the simpler data.

The other advantage is it makes your code more extensible in general.

If you don't want to do that, another example (since you're using the unittest module) would be to use unittest.mock.patch

In this case you would write your test as:

class TestExample(...):
    @patch('yourmodule.OPS', TEST_OPS)
    @patch('yourmodule.DIRECTIVES', TEST_DIRECTIVES)
    def text_example_option(self):
        # call example() and test against a known--good value
        # given TEST_OPS and TEST_DIRECTIVES

This is essentially the same thing, but gives a way to test your function against (temporarily assigned) new configurations without changing the defaults.

user_9822570
Related