Assertion improvements

user_2955048 0
Viewed 149

Currently in the company where I'm working, we have a framework to run tests. We want to integrate pytest to be able to write tests in the pytest way, but we need the old framework for all the things it's doing in the background.

The issue I'm facing is regarding assertions. Currently we have a bunch of assertion functions. All of them use a private method to write both to python logging and to a json file. I would like to get rid of them and use only "assert". What I did until now is to monkeypatch _pytest.assertion.rewrite.py with a custom module I created, where I changed the visit_Assert method and add this piece of code after line 873:

if isinstance(assert_.test, ast.Compare):
    test_value = BINOP_MAP[assert_.test.ops[0].__class__]
    test_type = "Comparison"
elif isinstance(assert_.test, ast.Call):
    test_value = str(assert_.test.func.id)
    test_type = "FunctionCall"

And then I call the same private method I mentioned above to save the results.

As you could guess I don't think it's the best way to do that: is there a better way?

I tried with the different hooks, but could not find the information I need (what is the comparison the assert is doing), especially because pytest is very good when the tests fail (it makes sense), but not so rich in information when the tests pass.

pytest python
1 Answers

It depends a bit on which version of Pytest you're using, since the hooks are under pretty active development. But in any relatively recent version, you could implement the hook pytest_assertrepr_compare, which is called to report custom error messages on asserts that fail. This method can be defined in conftest.py, and pytest will happily use that definition.

A method like this:

def pytest_assertrepr_compare(config, op, left, right):
    print("Call legacy method here")
    return None

Would instruct pytest that no custom error messages are required (that's the return None part), but it would allow you to call arbitrary code on assert failures.

As an example, running pytest on a dummy test file, test_foo.py with contents:

def test_foo():
    assert 0 == 1, "No bueno"

Should give the following output on your terminal:

================================================= test session starts ==================================================
platform darwin -- Python 3.9.0, pytest-6.1.2, py-1.9.0, pluggy-0.13.1 -- /usr/local/opt/python@3.9/bin/python3.9
cachedir: .pytest_cache
rootdir: /Users/bnaecker/tmp
plugins: cov-2.10.1
collected 1 item

foo.py::test_foo FAILED                                                                                          [100%]

======================================================= FAILURES =======================================================
_______________________________________________________ test_foo _______________________________________________________

    def test_foo():
>       assert 0 == 1, "No bueno"
E       AssertionError: No bueno
E       assert 0 == 1
E         +0
E         -1

foo.py:6: AssertionError
------------------------------------------------- Captured stdout call -------------------------------------------------
Call legacy method here
=============================================== short test summary info ================================================
FAILED foo.py::test_foo - AssertionError: No bueno
================================================== 1 failed in 0.10s ===================================================

The captured stdout is a stand-in for calling your custom logging function. Also, note I'm using pytest-6.1.2, and it's not clear when this hook was included. Other similar hooks were introduced in 5.0, so it's plausible that anything in the >=6.0 would be fine, but YMMV.

Rereading your question, it occurs that you might be more specifically asking about how to call your custom method when an assertion passes, rather than when it fails. In that case, the experimental method pytest_assertion_pass may be what you're looking for. The setup is the same, just implement that method instead in your conftest.py.

user_13548540
Related