PyTorch clip_grad_norm vs clip_grad_norm_, what is the differece when it has underline?

user_7339624 0
Viewed 372

When coding PyTorch in torch.nn.utils I see two functions, clip_grad_norm and clip_grad_norm_.

I want to know the difference so I went to check the documentation but when I searched I only found the clip_grad_norm_ and not clip_grad_norm.

So I'm here to ask if anyone knows the difference.

backpropagation pytorch deep-learning gradient
1 Answers

Pytorch uses the trailing underscore convention for in-place operations. So the difference is that the one with an underscore modifies the tensor in place and the other one leaves the original tensor unmodified and returns a new tensor.

user_27900470
Related