Why GPU memory usage is increasing when I call a method that uses tensors already exist on GPU?

Viewed 21

Consider the following overviews of two codes in PyTorch. Dataloader and all other aspects are the same for the two codes except the extra_class, which is present in some other module.

Code #1:

main(args)
......
train(nn)
......

Code #2:

main(args)
......
object = extra_class(...arguments...)       #creating object
train(nn)
object.method1()                           #calling method on object
......

Extra class in another module

extra_class(...parameters......)
    __init__(...):
         tensor1.to(device)  # using a lot of `.to(device)` on tensors
         tensor2.to(device) 
         ..................
   method1() { .... }  # using the tensors that are transferred to device in __init__()

     

When I run code #1, then there is a constant usage of GPU memory (say 70%). But when I run code #2, while executing the line object.method1(), the memory of GPU is increasing suddenly and after the line, it is decreasing. So, I can safely infer that the line is loading many tensors into GPU memory during execution, which is causing an increase in executing time. But I am not understanding why it is happening.

I am not using .to(device) in method1 of extra_class. The method1 is only using the tensors that are transferred to the GPU device in __init__() of extra_class only. So all the transfers have to be done during object creation only ( object = extra_class(...arguments...)) if I am correct, but why the transfer is happening when I call the method1 on the object?

1 Answers

It is an issue caused due not detaching a heavy tensor that is passed to the method.

If you do not detach the inputs from the computational graph in PyTorch, then it will construct a graph containing that variable also.

Related