I'm trying to backpropogate through a Tensor.index_add operation, but it breaks when index_add broadcasts vectors. Is this intended, or is this a bug in PyTorch?
#The following code does NOT throw an error:
m=torch.zeros(4)
m.requires_grad=True
i=[0,1,2,3]
v=[6,7,8,9]
i=torch.tensor(i).long()
v=torch.tensor(v).float()
v.requires_grad=True
print(m)
m=m.index_add(0,i,v)
print(m)
m.sum().backward()
print(v.grad)
print("DONE")
#The following code throws an error:
m=torch.zeros(4,4)
m.requires_grad=True
i=[0,1,2,3]
v=[6,7,8,9]
i=torch.tensor(i).long()
v=torch.tensor(v).float()
v.requires_grad=True
print(m)
m=m.index_add(0,i,v)
print(m)
m.sum().backward()
print(v.grad)
print("DONE")
That code, when run, gives the following output and error:
tensor([0., 0., 0., 0.], requires_grad=True)
tensor([6., 7., 8., 9.], grad_fn=<IndexAddBackward>)
tensor([1., 1., 1., 1.])
DONE
tensor([[0., 0., 0., 0.],
[0., 0., 0., 0.],
[0., 0., 0., 0.],
[0., 0., 0., 0.]], requires_grad=True)
tensor([[6., 6., 6., 6.],
[7., 7., 7., 7.],
[8., 8., 8., 8.],
[9., 9., 9., 9.]], grad_fn=<IndexAddBackward>)
ERROR: RuntimeError: expand(torch.FloatTensor{[4, 4]}, size=[4]): the number of sizes provided (1) must be greater or equal to the number of dimensions in the tensor (2)