How can I prevent the value of a fixture to be printed?

Viewed 540

I've created a pytest fixture which gets a token. When the tests which use this fixture fail, then the token will be printed in the logs. On the one hand that is not helpful, on the other hand it is a security issue.

How can I prevent the fixtures content to be printed?

MVCE

import pytest

@pytest.fixture
def token():
    yield "secret"

def test_foo(token):
    assert False

shows the "secret":

enter image description here

3 Answers

The easiest solution is to change the traceback format, e.g. pytest --tb=short will omit printing test function args. This can also be persisted in pytest.ini, effectively modifying the default pytest invocation:

[pytest]
addopts = --tb=short

However, you can also customize the output by extending pytest.

Technically, everything pytest prints to terminal is contained in TestReport, so you can modify the report object after the test finishes, but before the failure summary is printed. Example code, to be put in a conftest.py in the project or tests root dir:

def pytest_runtest_logreport(report):
    if report.longrepr is None:
        return
    for tb_repr, *_ in report.longrepr.chain:
        for entry in tb_repr.reprentries:
            if entry.reprfuncargs is not None:
                args = entry.reprfuncargs.args
                for idx, (name, value) in enumerate(args):
                    if name == "token":
                        args[idx] = (name, "********")
            if entry.reprlocals is not None:
                lines = entry.reprlocals.lines
                for idx, line in enumerate(lines):
                    if line.startswith("token"):
                        lines[idx] = "token          = '*********'"

Although clumsy and untested, this demonstrates the approach: get the traceback info stored in the report, if any entry has either reprfuncargs available (this contains values for all test function arguments, including fixtures), modify the token value if present. Do the same for reprlocals (those are the f_locals of the recorded frame and are printed when you invoke e.g. pytest --showlocals).

When running the test now, you should get the modified error output like

===== FAILURES =====
_____ test_foo _____

token = ********

    def test_foo(token):
>       assert False
E       assert False

The pytest_runtest_logreport hook is used to postprocess the report object created in pytest_runtest_makereport, before the actual reporting starts.

An approach which works well enough, adapted from here:

import pytest

class Secret:
    def __init__(self, value):
        self.value = value

    def __repr__(self):
        return "Secret(********)"

    def __str___(self):
        return "*******"

def get_from_vault(key):
    return "something looked up in vault"


@pytest.fixture(scope='session')
def password():
    return Secret(get_from_vault("key_in_value"))

def login(username, password):
    pass

def test_using_password(password):
    # reference the value directly in a function
    login("username", password.value)
    # If you use the value directly in an assert, it'll still log if it fails
    assert "something looked up in vault" == password.value

    # but won't be printed here
    assert False

This isn't perfect but it will be simpler. This is the output:

==================================================================================== FAILURES ====================================================================================
______________________________________________________________________________ test_using_password _______________________________________________________________________________

password = Secret(********)

    def test_using_password(password):
        # reference the value directly in a function
        login("username", password.value)
        # If you use the value directly in an assert, it'll still log if it fails
        assert "something looked up in vault" == password.value
    
        # but won't be printed here
>       assert False
E       assert False

test_stuff.py:31: AssertionError
============================================================================ short test summary info =============================================================================
FAILED test_stuff.py::test_using_password - assert False
Related