Consider the following overviews of two codes in PyTorch. Dataloader and all other aspects are the same for the two codes except the extra_class, which is present in some other module.
Code #1:
main(args)
......
train(nn)
......
Code #2:
main(args)
......
object = extra_class(...arguments...) #creating object
train(nn)
object.method1() #calling method on object
......
Extra class in another module
extra_class(...parameters......)
__init__(...):
tensor1.to(device) # using a lot of `.to(device)` on tensors
tensor2.to(device)
..................
method1() { .... } # using the tensors that are transferred to device in __init__()
When I run code #1, then there is a constant usage of GPU memory (say 70%). But when I run code #2, while executing the line object.method1(), the memory of GPU is increasing suddenly and after the line, it is decreasing. So, I can safely infer that the line is loading many tensors into GPU memory during execution, which is causing an increase in executing time. But I am not understanding why it is happening.
I am not using .to(device) in method1 of extra_class. The method1 is only using the tensors that are transferred to the GPU device in __init__() of extra_class only. So all the transfers have to be done during object creation only ( object = extra_class(...arguments...)) if I am correct, but why the transfer is happening when I call the method1 on the object?