Is there a way to include the param value and name in parametrized pytest tests in reports?

Viewed 30

I have a pytest that looks like this:

@pytest.mark.parametrize(
    "paramOne,paramTwo,paramThree,expected",
    [
        (True, False, False, "output one"),
        (False, True, False, "output two"),
        (False, False, True, "output three"),
    ],
)
def test__myfunc_method_should_do_something(
    paramOne,
    paramTwo,
    paramThree,
    expected,
):
    assert myfunc.method(paramOne,paramTwo,paramThree) == expected

These parameterized tests show up in my IDE (vscode) and reports like this:

test__myfunc_method_should_do_something
    True-False-False-output one
    False-True-False-output two
    False-False-True-output three

I would like it to look more like this though:

paramOne=True-paramTwo=False-paramThree=False-output one
paramOne=False-paramTwo=True-paramThree=False-output two
paramOne=False-paramTwo=False-paramThree=True-output three

Is there a way to include more metadata for how the parameterized tests are displayed, perhaps with ids?

I know I can manually name them like this:

@pytest.mark.parametrize(
    "paramOne,paramTwo,paramTwo,expected",
    [
        (True, False, False, "output one"),
        (False, True, False, "output two"),
        (False, False, True, "output three"),
    ],
        ids=["one", "two", "three"],
)

But is there a way to dynamically have ids just display the name and value of each param? Without me having to manually configure it for each one?

2 Answers

You can have a helper function to calculate the ids for you in the format of your choosing. This helper function can be shared throughout your code.

For example:

import pytest

param_names = "paramOne,paramTwo,paramThree,expected"
param_values = [
    (True, False, False, "output one"),
    (False, True, False, "output two"),
    (False, False, True, "output three"),
]


def pretty_ids(param_names, param_values):
    param_names = param_names.split(',')
    ids = []
    for param_value in param_values:
        single_id = []
        for index, value in enumerate(param_value):
            single_id.append(f"{param_names[index]}={value}")
        ids.append("-".join(single_id))
    return ids


@pytest.mark.parametrize(
    param_names,
    param_values,
    ids=pretty_ids(param_names, param_values)
)
def test__myfunc_method_should_do_something(
    paramOne,
    paramTwo,
    paramThree,
    expected,
):
    assert myfunc.method(paramOne,paramTwo,paramThree) == expected

There are a few different parts to this. One, changing the format displayed. And Two, altering the pytest summary report. Let's begin with the test modification to show how one can avoid the hard-coding of parameterized variables when printing them:

import pytest

@pytest.mark.parametrize(
    "paramOne,paramTwo,paramThree,expected",
    [
        (True, False, False, "output one"),
        (False, True, False, "output two"),
        (False, False, True, "output three"),
    ],
)
def test_myfunc(
    paramOne,
    paramTwo,
    paramThree,
    expected,
    request,
):
    # assert myfunc(paramOne,paramTwo,paramThree) == expected
    print("\nOld Name: %s" % request.node.name)

    arg_string = ""
    for item in request.fixturenames:
        if item != "request":
            arg_string += "%s=%s-" % (item, request.getfixturevalue(item))
    if arg_string.endswith("-"):
        arg_string = arg_string[:-1]
    print("New Name: %s" % arg_string)

I saved that to a file called test_stack_overflow_73746345.py and then ran it with pytest:

pytest test_stack_overflow_73746345.py 
====================== test session starts ======================
platform darwin -- Python 3.10.5, pytest-7.1.3, pluggy-1.0.0
...
collected 3 items                                                                                  

test_stack_overflow_73746345.py 
Old Name: test_myfunc[True-False-False-output one]
New Name: paramOne=True-paramTwo=False-paramThree=False-expected=output one
.
Old Name: test_myfunc[False-True-False-output two]
New Name: paramOne=False-paramTwo=True-paramThree=False-expected=output two
.
Old Name: test_myfunc[False-False-True-output three]
New Name: paramOne=False-paramTwo=False-paramThree=True-expected=output three
.

====================== 3 passed in 0.03s ======================

The output shows that the parameters become part of the default name, accessible via request.node.name if you include the request fixture in your test definition. The parameter names are accessible via request.fixturenames, and you can get the values via request.getfixturevalue(PARAMETER_NAME). That's part One.

Part Two is changing the pytest report summary. To do that, create a conftest.py file and drop the following code into it as a start:

def pytest_terminal_summary(terminalreporter):
    pass  # Make changes here using the terminalreporter arg

By calling methods from terminalreporter, you should be able to update your output based on the strings generated in the test, if you saved them somewhere accessible. If this is your first time making changes to the pytest terminal summary, there are some existing Stack Overflow solutions that can help you get started with that, such as https://stackoverflow.com/a/65944137/7058266, so that you can change your test summary as needed.

Related