Exclude some tests by default

user_5227053 0
Viewed 444

I wish to configure pytest such that it excludes some tests by default; but it should be easy to include them again with some command line option. I only found -k, and I have the impression that that allows complex specifications, but am not sure how to go about my specific need...

The exclusion should be part of the source or a config file (it's permanent - think about very long-running tests which should only be included as conscious choice, certainly never in a build pipeline...).

Bonus question: if that is not possible, how would I use -k to exclude specific tests? Again, I saw hints in the documentation about a way to use not as a keyword, but that doesn't seem to work for me. I.e., -k "not longrunning" gives an error about not being able to find a file "notrunning", but does not exclude anything...

pytest python
3 Answers

You can use pytest to mark some tests and use -k arg to skip or include them.

For example consider following tests,

import pytest

def test_a():
    assert True

@pytest.mark.never_run
def test_b():
    assert True


def test_c():
    assert True
    
@pytest.mark.never_run
def test_d():
    assert True

you can run pytest like this to run all the tests

pytest

To skip the marked tests you can run pytest like this,

pytest -m "not never_run"

If you want to run the marked tests alone,

pytest -m "never_run"
user_78968490

goal: by default skip tests marked as @pytest.mark.integration

conftest.py

import pytest

# function executed right after test items collected but before test run
def pytest_collection_modifyitems(config, items):
    if not config.getoption('-m'):
        skip_me = pytest.mark.skip(reason="use `-m integration` to run this test")
        for item in items:
            if "integration" in item.keywords:
                item.add_marker(skip_me)

pytest.ini

[pytest]
markers =
    integration: integration test that requires environment

now all tests marked as @pytest.mark.integration are skipped unless you use

pytest -m integration
user_12766640

What I have done in the past is create custom markers for these tests that way I can exclude them using the -m command line flag for running the tests. So for example in your pytest.ini file place the following content:

[pytest]
markers =
    longrunning: marks a test as longrunning

Then we just have to mark our long running tests with this marker.

@pytest.mark.longrunning
def test_my_long_test():
    time.sleep(100000)

Then when we run the tests we would do pytest -m "not longrunning" tests/ to exclude them and pytest tests to run everything as intended.

user_68178350
Related