How to use Pytest to test functions that generate random samples?

user_9571463 0
Viewed 4889

Lets say I generate some input numpy array data using a np.random.normal() in my test_func.py script that is using pytest.

Now I want to call the func.py function that I am testing. How am I able to get testable results? If I set a seed in the test_func.py script, it isn't going to correspond to the random data that gets generated in the func.py function, correct?

I want to be able to create some reference data in test_func.py and then test that the randomness generated in the func.py script is comparable to the reference data I created (hence, testing the randomness and functionality of the func.py function).

Thank you!

EDIT: Here is some sample code to describe my process:

# func.py
import numpy as np
# I send in a numpy array signal, generate noise, and append noise to signal
def generate_random_noise(signal):
    noise = np.random.normal(0, 5, signal.shape)
    signal_w_noise = signal + noise
    return signal_w_noise


# test_func.py
import pytest
import numpy as np
import func
def test_generate_random_noise():
    # create reference signal
    # ...
    np.random.seed(5)
    reference_noise = np.random.normal(0, 5, ref_signal.shape)
    ref_signal_w_noise = ref_signal + reference_noise

    # assert manually created signal and noise and 
    assert all(np.array_equal(x, y) for x, y in zip(generate_random_noise(reference_signal), ref_signal_w_noise))
pytest random-seed python-3.x random numpy
1 Answers

When using random stuff you can have 2 test approach:

  • Use a known seed to ensure the function depending on the random distribution performs as expected: using a known seed you can compare function behavior to a known in advance behavior.
  • Validate the statistical behavior of the function depending on the random distribution. Here you need to do some maths on the expected distribution of the function "results" and have some statistical metrics used as success/fail criteria eg are the mean, skew,... of the tested function matching their expected ojective. This can be done using a non frozen seed but a lot of functions calls need to be collected to have sufficient data to have meaningfull statistics.
user_60927880
Related