Why does `py.test --last-failed` skip many tests altough no tests had failed before?

user_543875 0
Viewed 3229

Out of sudden py.test --last-failed skips a lot of tests although no test failed in a previous test run. Here is the output of first running py.test --vv and then py.test --vv --last-failed:

$venv/bin/py.test -vv
======================================================================= test session starts =======================================================================
platform darwin -- Python 3.7.3, pytest-4.6.3, py-1.8.0, pluggy-0.12.0 -- /Users/lars/coding/talea/fahrtwind/products/backend/venv/bin/python3
cachedir: .pytest_cache
rootdir: /Users/lars/coding/talea/fahrtwind/products/backend, inifile: pytest.ini, testpaths: tests
plugins: forked-1.0.2, env-0.6.2, xdist-1.28.0, cov-2.7.1
collected 888 items
...                                                                                                                                     
<HERE ALL 888 TESTS ARE RUN AND ALL PASS>
...
=================================================================== 888 passed in 63.89 seconds ===================================================================
$
$
$venv/bin/py.test -vv --last-failed
======================================================================= test session starts =======================================================================
platform darwin -- Python 3.7.3, pytest-4.6.3, py-1.8.0, pluggy-0.12.0 -- /Users/lars/coding/talea/fahrtwind/products/backend/venv/bin/python3
cachedir: .pytest_cache
rootdir: /Users/lars/coding/talea/fahrtwind/products/backend, inifile: pytest.ini, testpaths: tests
plugins: forked-1.0.2, env-0.6.2, xdist-1.28.0, cov-2.7.1
collected 303 items                                                                                                                                               
run-last-failure: 181 known failures not in selected tests (skipped 206 files)
...
<HERE ONLY 303 TESTS ARE RUN>
...

Why does py.test --last-failed skip many tests altough no tests had failed before?

pytest python
1 Answers

As suggested in the comments, invalidating the cache was the solution: either rerun the tests via

$ pytest --cache-clear

or, if the test suite takes a lot of time to rerun, simply delete the .pytest_cache directory in the project root (or .cache for older versions of pytest). Also, citing the Clearing Cache content section:

You can instruct pytest to clear all cache files and values by adding the --cache-clear option like this:

pytest --cache-clear

This is recommended for invocations from Continuous Integration servers where isolation and correctness is more important than speed.

Thus, it is a good practice to instruct your CI server to invalidate the cache on each run so you have a reference when encountering unexpected failures due to local caching.

user_26502490
Related