Among thousands of test cases in a project I'm working on there are (about 10) tests which are very likely to fail (because they test something that can be broken easily). In order to make tests fail quickly I'd like to make them run first.
I know --failed-first and the ordering-plugin - both are helpful but don't fully meet my requirements: I'd like pytest to be aware of all the tests that failed in the past - not only in the last run and it would be great to let pytest discover those cases (rather than having to specify them manually.
So is there an approach do accomplish --failed-most-often-in-the-past?
How do you do this?
Current approach
By - manually - collecting failing tests you can run them first by just putting them in front of the ones you want to run:
time pytest -x tests/some/generic/test_file.py::failing_test tests
is much faster than
time (pytest -x tests/some/generic/test_file.py::failing_test; pytest -x tests)
You could now automatically collect failing tests and prepend them to a file:
pytest -x $(cat failing_tests.txt) tests