pytorch how to compute grad after clone a tensor

Viewed 529

My simple code:

import torch

x = torch.randn(4, requires_grad=True).cuda()
y = torch.randn(4, requires_grad=True).cuda()
z = torch.zeros(4)
z = torch.clone(x)
z.retain_grad()
h = (z + y) * z
l = torch.randn(4).cuda()
loss = (l - h).pow(2).sum()
loss.backward()
print('x.grad=', x.grad)
print('z.grad=', z.grad)

output:

x.grad= None
z.grad= tensor([-15.3401,  -3.2623,  -2.1670,   0.1410], device='cuda:0')

Why x.grad is None but not same as z.grad ? What should I do if I want they are same?

1 Answers

You need to call x.retain_grad() after declaring x if you want to keep the grad of tensor x.

Related