Is there a way to programatically catch a cuda out of memory runtime error and then restart and run all? My model can run about 5 runs before cuda runs out of memory and i'm having to manually stop the process and restart & run all cells.
Is there a way to programatically catch a cuda out of memory runtime error and then restart and run all? My model can run about 5 runs before cuda runs out of memory and i'm having to manually stop the process and restart & run all cells.