`pytest`: Downloading a test file once, and using it for multiple tests

user_1610428 0
Viewed 1570

I am using pytest to run test case for a package I am developing. The tests use a small image file that I have saved as a github asset. The code below works just fine, but I think that pytest is downloading the image each time it runs a new test and that takes unnecessary time and resources. I was trying to figure out how I can download the file once, and then share it across test cases

Here is some sample code.

# -- in conftest.py --
import sys
import pytest
import os
import shutil
import requests

@pytest.fixture(scope="function")
def small_image(tmpdir):
    url = 'https://github.com/.../sample_image_small.tif'

    r = requests.get(url)

    with open(os.path.join(str(tmpdir), 'sample_image_small.tif'), 'wb') as f:
        f.write(r.content)

    return os.path.join(str(tmpdir), 'sample_image_small.tif')

Then here are some very simple test cases that should be able to share the same image.

# -- test_package.py --
import pytest
import os

@pytest.mark.usefixtures('small_image')


def test_ispath(small_image, compression):

    assert os.path.exists(small_image)

def test_isfile(small_image, compression):

    assert os.path.isfile(small_image)

Now I believe that pytest will try and isolate each test by itself and so that is what causes the repeated downloads of files. I tried to set the @pytest.fixture(scope="module") instead of function but that was generating strange errors:

ScopeMismatch: You tried to access the 'function' scoped fixture 'tmpdir' with a 'module' scoped request object, involved factories

Is there a better way to setup the tests so that I don't keep download the file over and over?

pytest fixtures testing python
2 Answers

First, a note beforehand: a better alternative to the old tmpdir/tmpdir_factory fixtures pair is tmp_path/tmp_path_factory which deals with pathlib objects instead of the deprecated py.path, see Temporary directories and files.

Second, if you want to handle files session-scoped (or module-scoped), tmp*_factory fixtures are meant for that. Example:

@pytest.fixture(scope='session')
def small_image(tmp_path_factory):
    img = tmp_path_factory.getbasetemp() / 'sample_image_small.tif'
    img.write_bytes(b'spam')
    return img

The sample_image_small.tif will now be written once per test run.

Of course, there's nothing wrong with using tempfile as suggested by @MrBean Bremen in his answer, this is just an alternative doing the same, but using only standard pytest fixtures.

user_26502490

You can use the same code, just handle the tempfile yourself instead of using the tmpdir fixture (which cannot be used in module-scoped fixtures):

import os
import tempfile
import pytest
import requests

@pytest.fixture(scope="module")
def small_image():
    url = 'https://github.com/.../sample_image_small.tif'

    r = requests.get(url)
    f = tempfile.NamedTemporaryFile(delete=False):
    f.write(f.content)
    yield f.name
    os.remove(f.name)

This will create the file, return the file name, and delete the file after the tests are finished.

EDIT: The answer by @hoefling shows a more standard way to do this, I'll leave this one for reference.

user_124807300
Related