Run Pytest Classes in Custom Order

user_11880235 0
Viewed 926

I am writing tests with pytest in pycharm. The tests are divided into various classes.
I would like to specify certain classes that have to run before other classes.
I have seen various questions on stackoverflow (such as specifying pytest tests to run from a file and how to run a method before all other tests).
These and various others questions wanted to choose specific functions to run in order. This can be done, I understand, using fixtures or with pytest ordering.
I don't care which functions from each class run first. All I care about is that the classes run in the order I specify. Is this possible?

pytest pycharm function class python
1 Answers

Approach

You can use the pytest_collection_modifyitems hook to modify the order of collected tests (items) in place. This has the additional benefit of not having to install any third party libraries.

With some custom logic, this allows to sort by class.

Full example

Say we have three test classes:

  1. TestExtract
  2. TestTransform
  3. TestLoad

Say also that, by default, the test order of execution would be alphabetical, i.e.:

TestExtract -> TestLoad -> TestTransform

which does not work for us due to test class interdependencies.

We can add pytest_collection_modifyitems to conftest.py as follows to enforce our desired execution order:

# conftest.py
def pytest_collection_modifyitems(items):
    """Modifies test items in place to ensure test classes run in a given order."""
    CLASS_ORDER = ["TestExtract", "TestTransform", "TestLoad"]
    class_mapping = {item: item.cls.__name__ for item in items}

    sorted_items = items.copy()
    # Iteratively move tests of each class to the end of the test queue
    for class_ in CLASS_ORDER:
        sorted_items = [it for it in sorted_items if class_mapping[it] != class_] + [
            it for it in sorted_items if class_mapping[it] == class_
        ]
    items[:] = sorted_items

Some comments on the implementation details:

  • Test classes can live in different modules
  • CLASS_ORDER does not have to be exhaustive. You can reorder just those classes on which to want to enforce an order (but note: if reordered, any non-reordered class will execute before any reordered class)
  • The test order within the classes is kept unchanged
  • It is assumed that test classes have unique names
  • items must be modified in place, hence the final items[:] assignment
user_89298550
Related