TypeError: backward() got an unexpected keyword argument 'grad_tensors' in pytorch

user_5865579 0
Viewed 988

I have the following

w = torch.tensor([1.], requires_grad=True)
x = torch.tensor([2.], requires_grad=True)

a = torch.add(w, x)
b = torch.add(w, 1)

y0 = torch.mul(a, b)    # y0 = (x+w) * (w+1)
y1 = torch.add(a, b)    # y1 = (x+w) + (w+1)     


loss = torch.cat([y0, y1], dim=0)       # [y0, y1]

weight = torch.tensor([1., 2.])

loss.backward(grad_tensors=weight)

The above give me TypeError: backward() got an unexpected keyword argument 'grad_tensors' I check the website , the grad_tensors does live in the backward.

However, when I use

loss.backward(gradient=weight)

It works. gradient is not a parameters in backward. Any idea of that? my pytorch version is 1.7.0. Thanks.

pytorch gradient python-3.x
1 Answers

You are calling the torch.Tensor.backward, not torch.autograd.backward.

As for your second question about the difference b/w the two, torch.Tensor.backward internally calls torch.autograd.backward, which calculates gradients of given tensors w.r.t. graph leaves.

torch.autograd.backward(self, gradient, retain_graph, create_graph)

which corresponds to

torch.autograd.backward(tensors: self, grad_tensors: gradient, retain_graph, create_graph)

Thus, below two are equivalent:

loss.backward(gradient=weight)
torch.autograd.backward(loss, weight)
user_62108070
Related