Parameterize List Returned From Fixture in Pytest Or Pytest-cases

user_1322962 0
Viewed 114

I'm attempting to write a test fixture based on randomly generated data. This randomly generated data needs to be able to accept a seed so that we can generate the same data on two different computers at the same time.

I'm using the pytest parse.addoption fixture (I think it's a fixture) to add this ability.

My core issue is that I'd like to be able to parameterize a randomly generated list that uses a fixture as an argument.

from secrets import randbelow

from pytest_cases import parametrize_with_cases, fixture, parametrize

def pytest_addoption(parser):
    parser.addoption("--seed", action="store", default=randbelow(10))

@fixture(scope=session)
def seed(pytestconfig):
    return pytestconfig.getoption("seed")

@fixture(scope=session)
def test_context(seed):
    # In my actual tests these are randomly generated from the seed.
    # each element here is actually a dictionary but I'm showing strings
    # for simplicity of example.
    return ['a', 'test', 'list']


@parametrize(group_item=test_context["group_items"])
def case_group_item(group_item: str): 
    return group_item, "expected_result_goes_here"


@parametrize_with_cases("sql_statement, expected_result", cases='.')
def test_example(
        sql_statement: str,
        expected_result: int) -> None:
    assert False

Leads to this result.

% pytest test.py
========================================================================================================================================================================== test session starts ===========================================================================================================================================================================
platform darwin -- Python 3.8.2, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /Users/{Home}/tests, configfile: pytest.ini
plugins: datadir-1.3.1, celery-4.4.7, anyio-3.4.0, cases-3.6.11
collected 0 items / 1 error

================================================================================================================================================================================= ERRORS =================================================================================================================================================================================
________________________________________________________________________________________________________________________________________________________________________ ERROR collecting test.py ________________________________________________________________________________________________________________________________________________________________________
test.py:12: in <module>
    ???
E   TypeError: 'function' object is not subscriptable
======================================================================================================================================================================== short test summary info =========================================================================================================================================================================
ERROR test.py - TypeError: 'function' object is not subscriptable
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================================================================================================================ 1 error in 0.18s ============================================================================================================================================================================

I think I might be able work around this issue by making an empty test that leaks the test_context up to the global scope but that feels really really brittle. I'm looking for another method to still be able to

  1. Use the seed fixture to generate data
  2. Generate one test per element in the generated list
  3. Not depend on the order in which the tests are run.

Edit

Here's an example of this not working with straight pytest

import pytest

from pytest_cases import parametrize_with_cases, fixture, parametrize


@fixture
def seed():
    return 1

@fixture
def test_context(seed):
    return [seed, 'a', 'test', 'list']

@pytest.fixture(params=test_context)
def example_fixture(request):
    return request.param

def test_reconciliation(example_fixture) -> None:
    print(example_fixture)
    assert False

pytest test.py
========================================================================================================================================================================== test session starts ===========================================================================================================================================================================
platform darwin -- Python 3.8.2, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /Users/{HOME}/tests/integration, configfile: pytest.ini
plugins: datadir-1.3.1, celery-4.4.7, anyio-3.4.0, cases-3.6.11
collected 0 items / 1 error

================================================================================================================================================================================= ERRORS =================================================================================================================================================================================
________________________________________________________________________________________________________________________________________________________________________ ERROR collecting test.py ________________________________________________________________________________________________________________________________________________________________________
test.py:14: in <module>
    ???
../../../../../.venvs/data_platform/lib/python3.8/site-packages/_pytest/fixtures.py:1327: in fixture
    fixture_marker = FixtureFunctionMarker(
<attrs generated init _pytest.fixtures.FixtureFunctionMarker>:5: in __init__
    _inst_dict['params'] = __attr_converter_params(params)
../../../../../.venvs/data_platform/lib/python3.8/site-packages/_pytest/fixtures.py:1159: in _params_converter
    return tuple(params) if params is not None else None
E   TypeError: 'function' object is not iterable
======================================================================================================================================================================== short test summary info =========================================================================================================================================================================
ERROR test.py - TypeError: 'function' object is not iterable
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================================================================================================================ 1 error in 0.23s ======================================================================================================================================================================
pytest-cases pytest python
3 Answers

I tried your code with testfile and conftest.py

conftest.py

import pytest
from secrets import randbelow
from pytest_cases import parametrize_with_cases, fixture, parametrize


def pytest_addoption(parser):
    # If you add a breakpoint() here it'll never be hit.
    parser.addoption("--seed", action="store", default=randbelow(1))

@fixture(scope="session")
def seed(pytestconfig):
    # This line throws an exception since seed was never added.
    return pytestconfig.getoption("seed")

myso_test.py

import pytest
from pytest_cases import parametrize_with_cases, fixture, parametrize

@fixture(scope="session")
def test_context(seed):
    # In my actual tests these are randomly generated from the seed.
    # each element here is actually a dictionary but I'm showing strings
    # for simplicity of example.
    return ['a', 'test', 'list']

@parametrize("group_item", [test_context])
def case_group_item(group_item: str): 
    return group_item, "expected_result_goes_here"


@parametrize_with_cases("sql_statement, expected_result", cases='.')
def test_example(
        sql_statement: str,
        expected_result: int) -> None:
    assert True

Test Run:

PS C:\Users\AB45365\PycharmProjects\SO> pytest .\myso_test.py -s -v --seed=10 
============================================================== test session starts ==============================================================
platform win32 -- Python 3.9.2, pytest-6.2.5, py-1.10.0, pluggy-1.0.0 -- c:\users\ab45365\appdata\local\programs\python\python39\python.exe       
cachedir: .pytest_cache
rootdir: C:\Users\AB45365\PycharmProjects\SO
plugins: cases-3.6.11, lazy-fixture-0.6.3
collected 1 item

myso_test.py::test_example[group_item-test_context] PASSED
user_58338650

To complement Devang Sanghani's answer : as of pytest 7.1, pytest_addoption is a pytest plugin hook. So, as for all other plugin hooks, it can only be present in plugin files or in conftest.py.

See the note in https://docs.pytest.org/en/7.1.x/reference/reference.html#pytest.hookspec.pytest_addoption :

This function should be implemented only in plugins or conftest.py files situated at the tests root directory due to how pytest discovers plugins during startup.

This issue is therefore not related to pytest-cases.

user_72622470

After doing some more digging I ran into this documentation around pytest-cases

from secrets import randbelow

import pytest
from pytest_cases import parametrize_with_cases, fixture, parametrize

def pytest_addoption(parser):
    parser.addoption("--seed", action="store", default=randbelow(1))

@fixture(scope="session")
def seed(pytestconfig):
    # return pytestconfig.getoption("seed")
    return 1

@pytest.fixture(scope="session")
def test_context(seed):
    # In my actual tests these are randomly generated from the seed.
    # each element here is actually a dictionary but I'm showing strings
    # for simplicity of example.
    return ['a', 'test', 'list']


@parametrize("group_item", [test_context])
def case_group_item(group_item: str): 
    return group_item, "expected_result_goes_here"


@parametrize_with_cases("sql_statement, expected_result", cases='.')
def test_example(
        sql_statement: str,
        expected_result: int) -> None:
    assert False

This unfortunately ran me into a new problem. Looks like pytest-cases doesn't currently call pytest_addoption during the fixture execution step rihgt now. I created this ticket to cover this case but this does effectively solve my original question even if it has a caveat.

user_13229620
Related