Jupyter Notebook GPU memory release after training model

user_15841374 0
Viewed 495

How could we clear up the GPU memory after finishing a deep learning model training with Jupyter notebook. The problem is, no matter what framework I am sticking to (tensorflow, pytorch) the memory stored in the GPU do not get released except I kill the process manually or kill the kernel and restart the Jupyter. Do you have any idea how we can possible get rid of this problem by automating the steps?

jupyter-notebook deep-learning memory-management gpu python
1 Answers

The only walkaround I found was to use threading. Executing the Training with a subprocess.

An example:

def Training(arguments):
    ....
    ....
    return model   

if __name__=='__main__':
Subprocess = Process(# The complete function defined above
                     target = Training,
                     # Pass the arguments defined in the complete function 
                     # above.
                     # Note - The comma after the arguments : In order for 
                     # Python
                     # to understand this is a tuple
                     args = (arguments, ))

# Starting the defined subprocess
Subprocess.start()
# Wait for the subprocess to get completed
Subprocess.join()
user_158413740
Related