pytest - how to store that certain test failed with test name

vanguard 0
Viewed 29

I would like to store that specific tests failed, then pass that info via API when test class is finished.

I tried sth like that:

fails = []
@pytest.fixture(scope='function')
def something(request):
    yield
    if request.session.testsfailed:
        print("I failed")
        fails.append(request.node.name)
    print('FAILED', fails)

class TestLala:

    @pytest.mark.order(1)
    def test_test1(self, something):
        assert False

    @pytest.mark.order(3)
    def test_test1(self, something):
        assert True

    @pytest.mark.order(3)
    def test_test3(self, something):
        assert 4 == 4

but instead of failed tests I am still getting each test name added to the list.

pytest continuous-integration jenkins testing python
2 Answers

The pytest_runtest_makereport hook should solve it for you. Add this to your conftest.py

import pytest

pytest.failed_nodes = []

@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item):
    output = yield
    report = output.get_result()

    if report.failed:
        node_id = report.nodeid
        pytest.failed_nodes.append(node_id)
Teejay Bruno0

##EDIT##

The answer @Teejay Bruno gave shows how you can avoid having the push to your API run after every function. Just send the data to a list of dict after every test. Then call the send_data when the test is over to send to your API

I have done something similar in my work. By using conftest with make_report you are able to capture the test results and other meta data, then do as you please with that data (like send to a database or API).

test.py

import pytest


class TestLala:

    @pytest.mark.order(1)
    def test_test1(self):
        assert False

    @pytest.mark.order(3)
    def test_test1(self):
        assert True

    @pytest.mark.order(3)
    def test_test3(self):
        assert 4 == 4

conftest.py

import pytest

test = None
status_tag = None
line = None
duration = None
exception = None


@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    global test, status_tag, line, duration, exception
    report = yield
    result = report.get_result()

    if result.when == 'call':
        (filename, line, name) = item.location
        test = item.nodeid
        status_tag = result.outcome
        line = line
        duration = call.duration
        exception = call.excinfo


@pytest.fixture(scope='function', autouse=True)
def send_data(pytestconfig):
    yield
    global test, status_tag, line, duration, exception

    # This is where you can send the data to your API
    # This will run after every test so if you dont want to send the data as it comes in, you will need to change
    # how this function and the one above work a little
    print(f"TEST: {test}")
    print(f"STATUS_TAG: {status_tag}")
    print(f"LINE: {line}")
    print(f"DURATION: {duration}")
    print(f"EXCEPTION: {exception}")
    test = None
    status_tag = None
    line = None
    duration = None
    exception = None

If you have not worked with conftest before see the below link: https://docs.pytest.org/en/6.2.x/fixture.html

Search for the section titled -> "conftest.py: sharing fixtures across multiple files"

Ryan Burch0
Related