Django Pytest parallel run - Database does not exist

user_1199807 0
Viewed 646

I am using pytest to run my django tests and was trying to speed them up by running them in parallel. I tried modifying my database name using the enviroment variable PYTEST_XDIST_WORKER but my tests still cannot access the database.

How do I need to set my database settings in my django configuration so that the parallel databases are created and used.

db_suffix = env.str("PYTEST_XDIST_WORKER", "")
DATABASES["default"]["NAME"] = f"""{DATABASES["default"]["NAME"]}_{db_suffix}"""

pytest.ini

[pytest]
addopts = --ds=config.settings.test --create-db
python_files = tests.py test_*.py
norecursedirs = node_modules
env_files=
    .env.tests

command to run tests: pytest -n 4

Error: database "auto_tests_gw0" does not exist

pytest django python
2 Answers

I found the solution to this problem in this article https://iurisilv.io/2021/03/faster-parallel-pytest-django.html. You need to override the database suffix fixture.

@pytest.fixture(scope="session")
def django_db_modify_db_settings_xdist_suffix(request):
    skip_if_no_django()

    xdist_suffix = getattr(request.config, "workerinput", {}).get("workerid")
    if xdist_suffix:
        # 'gw0' -> '1', 'gw1' -> '2', ...
        suffix = str(int(xdist_suffix.replace("gw", "")) + 1)
        _set_suffix_to_test_databases(suffix=suffix)
user_11998070

You can first make sure that you are not overriding the django_db_setup in your conftest.py https://pytest-django.readthedocs.io/en/latest/database.html#django-db-setup

In my case, I reduced the frequency of its occurence by reducing the number of CPUs from -n auto (which uses 12 on my server) to -n 6, but I see that you are running already with only 4 CPUs. Maybe reduce that to -n 2 to test.

user_67592130
Related