Varying parameters for a session-scoped pytest fixture

Question

I have a test suite with an expensive fixture (it spins up a bunch of containers in a cluster), so I'd like to use a session-scoped fixture for it. However, it's configurable on several axes, and different subsets of tests need to test different subsets of the configuration space.

Here's a minimal demonstration of what I'm trying to do. By default tests need to test the combinations x=1,y=10 and x=2,y=10, but the tests in test_foo.py need to test x=3,y=10 so override the x fixture:

conftest.py:

import pytest


@pytest.fixture(scope="session", params=[1, 2])
def x(request):
    return request.param


@pytest.fixture(scope="session", params=[10])
def y(request):
    return request.param


@pytest.fixture(scope="session")
def expensive(x, y):
    return f"expensive[{x}, {y}]"

test_bar.py:

def test_bar(expensive):
    assert expensive in {"expensive[1, 10]", "expensive[2, 10]"}

test_foo.py:

import pytest


@pytest.fixture(scope="session", params=[3])
def x(request):
    return request.param


def test_foo(expensive):
    assert expensive in {"expensive[3, 10]"}

When I run this, I get the following:

test_bar.py::test_bar[1-10] PASSED                                       [ 33%]
test_foo.py::test_foo[3-10] FAILED                                       [ 66%]
test_bar.py::test_bar[2-10] PASSED                                       [100%]

=================================== FAILURES ===================================
________________________________ test_foo[3-10] ________________________________

expensive = 'expensive[1, 10]'

    def test_foo(expensive):
>       assert expensive in {"expensive[3, 10]"}
E       AssertionError: assert 'expensive[1, 10]' in {'expensive[3, 10]'}

It appears to have reused the 1-10 fixture from test_bar for the 3-10 test in test_foo. Is that expected (some sort of matching by position in the parameter list rather than value), or a bug in pytest? Is there some way I can get it to do what I'm aiming for?

Incidentally, if I make x in test_foo.py non-parametric (just returning a hard-coded 3) it also fails, but in a slightly different way: it runs both test_bar tests first, then reuses the second fixture for the test_foo test.

user_12480730 · Answer

The problem here is that the expansive fixture is in session scope, and the parameters are only read once.

A workaround would be to move expansive to module scope, so it will be evaluated for each module. That would work, but it would evaluate the fixture for each test module, even if several of them use the same parameters. If you have several test modules that use the same parameters, you could use 2 separate fixtures with different scope instead, with the common code moved out, e.g:

import pytest

...

def do_expansive_stuff(a, b):  # a and b are normal arguments, not fixtures
   # setup containers with parameters a, b...
    return f"expensive[{a}, {b}]"

@pytest.fixture(scope="session")
def expensive(x, y):
    yield do_expansive_stuff(x, y)

@pytest.fixture(scope="module")
def expensive_module(x, y):
    yield do_expansive_stuff(x, y)

You would use expensive_module in test_foo. This is of course assuming that the expansive setup has to be done separately for each parameter set. The downside of this approach is of course having to use differently named fixtures.

It would be nice if someone would come up with a cleaner approach...

Varying parameters for a session-scoped pytest fixture

1 Answers