TypeError: can't multiply sequence by non-int type of 'tuple' in pytorch

user_10489227 0
Viewed 812

The code as below:

class L2Norm(nn.Module):
    def __init__(self):
        super(L2Norm, self).__init__()
        self.eps = 1e-10
    def forward(self, x):
        norm = torch.sqrt(torch.sum(x * x, dim = 1) + self.eps)
        x = x / norm.unsqueeze(-1).expand_as(x)
        return x

I want to normalize the features. The input x is the output of nn.Linear() in FC layer (x = self.fc1(x)). 

    def forward(self, x):
        x = self.layer1(x)
        x = self.layer2(x)
        x = self.layer3(x)
        x = self.layer4(x)

        x = x.view(x.size(0), -1)
        x = self.fc1(x)
        if self.feature:
            return x
#        x = self.last_bn(x)
        x = self.fc2(x)

        return x

I print the x as below:

(tensor([[-0.8409,  8.6126, -1.6639,  ..., -3.3563, 10.0872,  2.4730],
        [-1.3959,  0.5608, -0.9233,  ...,  0.4385, -0.7089, -1.3401],
        [ 0.5742, -3.8479,  1.7756,  ..., -4.2798, -5.0684, -0.9032],
        ...,
        [ 0.9205,  3.1602, -3.9247,  ..., -2.1396,  4.0262,  2.8075],
        [-0.2024,  0.5603,  0.0491,  ..., -0.1716, -0.2513,  0.1179],
        [ 4.8053,  0.3062, -1.6867,  ..., -1.5749,  0.5193,  0.8671]],
       device='cuda:0', grad_fn=<GatherBackward>), tensor([[-156.3423, -145.1505, -156.6586,  ..., -157.9570, -141.1895,
         -155.2964],
        [ -31.9854,  -30.2333,  -31.1459,  ...,  -30.3290,  -30.8740,
          -31.8696],
        [-141.5926, -144.1404, -141.1151,  ..., -144.7264, -145.9508,
         -141.7867],
        ...,
        [-193.6224, -192.4931, -195.6285,  ..., -194.2939, -191.7269,
         -192.7527],
        [  -5.8791,   -4.5035,   -5.6987,  ...,   -5.8316,   -5.9696,
           -5.6506],
        [ -77.5002,  -83.8829,  -84.7204,  ...,  -84.6169,  -83.8326,
          -83.6949]], device='cuda:0', grad_fn=<GatherBackward>))

However, the mistake occurs in the torch.sum(x*x, dim=1). I have any solution for the mistake.

pytorch python
2 Answers

x is a tuple of two tensors, as shown in your output. x * x would require a way to multiply two tuples.

If I simply define x as a tuple of ints, e.g. x=(1, 1), and tried the same code: x * x, I get the same error:

>>> x=(1,1)
>>> x * x
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can't multiply sequence by non-int of type 'tuple'

# though tuple times an int does work:
>>> x * 3
(1, 1, 1, 1, 1, 1)

In your case, x should probably be a tensor, not a tuple.

user_3495400

Thank everyone. I solved this problem. But I don't know why the features of FC layer are tuple. Finally I find the x=x[0] is the counterpart of tensor, which is also where the error occurs. And the x[0] returns tensor.

user_104892270
Related