My simple code:
import torch
x = torch.randn(4, requires_grad=True).cuda()
y = torch.randn(4, requires_grad=True).cuda()
z = torch.zeros(4)
z = torch.clone(x)
z.retain_grad()
h = (z + y) * z
l = torch.randn(4).cuda()
loss = (l - h).pow(2).sum()
loss.backward()
print('x.grad=', x.grad)
print('z.grad=', z.grad)
output:
x.grad= None
z.grad= tensor([-15.3401, -3.2623, -2.1670, 0.1410], device='cuda:0')
Why x.grad is None but not same as z.grad ? What should I do if I want they are same?