pytest - Change variable values in cells of Jupyter notebooks during execution

user_3285138 0
Viewed 288

We are planning to add an integration test to our Jupyter notebooks to verify that they run end-to-end by using pytest and the nbval plugin. One feature that we would like to have is the ability to parametarize notebook executions, so that we can try different values for variables in the Jupyter notebook cells and effectively test multiple scenarios for the same notebook.

E.g.

foo = 1

print(foo)

We would like to be able to assign any value like 1,2,3,100,900, etc to foo and see the output of execution. Essentially something like:
<EXECUTE_NOTEBOOK> -p foo=2
<EXECUTE_NOTEBOOK> -p foo=3
<EXECUTE_NOTEBOOK> -p foo=4
<EXECUTE_NOTEBOOK> -p foo=100
<EXECUTE_NOTEBOOK> -p foo=900

and see the different outputs.

We know that papermill already supports something like this: https://papermill.readthedocs.io/en/latest/usage-execute.html#execute-via-the-python-api

Does pytest have a feature like this as well?

jupyter-notebook pytest python
1 Answers

You could go with Papermill as one option, however you could also try parametrizing your tests inside the notebook itself, to avoid running each param set as a separate test (as in the example above). One way to do this is with the ipytest library, which is similar to nbval, but in my opinion a bit more straightforward (here is the example notebook).

If you basically have one input value that you want to manipulate and that will then determine the output of the whole notebook, one idea for what you could do is:

  1. wrap all the other functions/steps in the notebook into one function in the end
  2. use cell magic %%run_pytest[clean] so that ipython knows to run this cell
  3. decorate it with @pytest.mark.parametrize to pass the values you want to test

This way your whole notebook will be rerun a few times based on the params passed to this function.

user_103547890
Related