How to add custom sections to terminal report in pytest

user_1603480 0
Viewed 2222

In pytest, when a test case is failed, you have in the report the following categories:

  • Failure details
  • Captured stdout call
  • Captured stderr call
  • Captured log call

pytest report

I would like to add some additional custom sections (I have a server that turns in parallel and would like to display the information logged by this server in a dedicated section).

How could I do that (if ever possible)?

Thanks

NOTE:

I have currently found the following in source code but don't know whether that shall be right approach

nodes.py

class Item(Node):
    ...
    def add_report_section(self, when, key, content):
        """
        Adds a new report section, similar to what's done internally
        to add stdout and stderr captured output:: 
        ...
        """

reports.py

class BaseReport:
    ...

    @property
    def caplog(self):
        """Return captured log lines, if log capturing is enabled

        .. versionadded:: 3.5
        """
        return "\n".join(
            content for (prefix, content) in self.get_sections("Captured log")
        )
pytest python
2 Answers

To add custom sections to terminal output, you need to append to report.sections list. This can be done in pytest_report_teststatus hookimpl directly, or in other hooks indirectly (via a hookwrapper); the actual implementation heavily depends on your particular use case. Example:

# conftest.py

import os
import random
import pytest

def pytest_report_teststatus(report, config):
    messages = (
        'Egg and bacon',
        'Egg, sausage and bacon',
        'Egg and Spam',
        'Egg, bacon and Spam'
    )

    if report.when == 'teardown':
        line = f'{report.nodeid} says:\t"{random.choice(messages)}"'
        report.sections.append(('My custom section', line))


def pytest_terminal_summary(terminalreporter, exitstatus, config):
    reports = terminalreporter.getreports('')
    content = os.linesep.join(text for report in reports for secname, text in report.sections)
    if content:
        terminalreporter.ensure_newline()
        terminalreporter.section('My custom section', sep='-', blue=True, bold=True)
        terminalreporter.line(content)

Example tests:

def test_spam():
     assert True

def test_eggs():
     assert True


def test_bacon():
     assert False

When running the tests, you should see My custom section header at the bottom colored blue and containing a message for every test:

collected 3 items

test_spam.py::test_spam PASSED
test_spam.py::test_eggs PASSED
test_spam.py::test_bacon FAILED

============================================= FAILURES =============================================
____________________________________________ test_bacon ____________________________________________

    def test_bacon():
>        assert False
E        assert False

test_spam.py:9: AssertionError
---------------------------------------- My custom section -----------------------------------------
test_spam.py::test_spam says:   "Egg, bacon and Spam"
test_spam.py::test_eggs says:   "Egg and Spam"
test_spam.py::test_bacon says:  "Egg, sausage and bacon"
================================ 1 failed, 2 passed in 0.07 seconds ================================
user_26502490

The other answer shows how to add a custom section to the terminal report summary, but it's not the best way for adding a custom section per test.

For this goal, you can (and should) use the higher-level API add_report_section of an Item node (docs). A minimalist example is shown below, modify it to suit your needs. You can pass state from the test instance through an item node, if necessary.

In test_something.py, here is one passing test and two failing:

def test_good():
    assert 2 + 2 == 4

def test_bad():
    assert 2 + 2 == 5

def test_ugly():
    errorerror

In conftest.py, setup a hook wrapper:

import pytest

content = iter(["first", "second", "third"])

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_call(item):
    outcome = yield
    item.add_report_section("call", "custom", next(content))

The report will now display custom sections per-test:

$ pytest
============================== test session starts ===============================
platform linux -- Python 3.9.0, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /tmp/example
collected 3 items                                                                

test_something.py .FF                                                      [100%]

==================================== FAILURES ====================================
____________________________________ test_bad ____________________________________

    def test_bad():
>       assert 2 + 2 == 5
E       assert (2 + 2) == 5

test_something.py:5: AssertionError
------------------------------ Captured custom call ------------------------------
second
___________________________________ test_ugly ____________________________________

    def test_ugly():
>       errorerror
E       NameError: name 'errorerror' is not defined

test_something.py:8: NameError
------------------------------ Captured custom call ------------------------------
third
============================ short test summary info =============================
FAILED test_something.py::test_bad - assert (2 + 2) == 5
FAILED test_something.py::test_ugly - NameError: name 'errorerror' is not defined
========================== 2 failed, 1 passed in 0.02s ===========================
user_6740390
Related