How to convert a list of different-sized tensors to a single tensor?

user_13205338 0
Viewed 6385

I want to convert a list of tensors with different sizes to a single tensor.

I tried torch.stack, but it shows an error.

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-237-76c3ff6f157f> in <module>
----> 1 torch.stack(t)

RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 5 and 6 in dimension 1 at C:\w\1\s\tmp_conda_3.7_105232\conda\conda-bld\pytorch_1579085620499\work\aten\src\TH/generic/THTensor.cpp:612

My list of tensors:

[tensor([-0.1873, -0.6180, -0.3918, -0.5849, -0.3607]),
 tensor([-0.6873, -0.3918, -0.5849, -0.9768, -0.7590, -0.6707]),
 tensor([-0.6686, -0.7022, -0.7436, -0.8231, -0.6348, -0.4040, -0.6074, -0.6921])]

I have also tried this in a different way, instead of tensors, I used a lists of these individual tensors and tried to make a tensor out of it. That also showed an error. 

list: [[-0.18729999661445618, -0.6179999709129333, -0.3917999863624573, -0.5849000215530396, -0.36070001125335693], [-0.6873000264167786, -0.3917999863624573, -0.5849000215530396, -0.9768000245094299, -0.7590000033378601, -0.6707000136375427], [-0.6686000227928162, -0.7021999955177307, -0.7436000108718872, -0.8230999708175659, -0.6348000168800354, -0.40400001406669617, -0.6074000000953674, -0.6920999884605408]]

The error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-245-489aea87f307> in <module>
----> 1 torch.FloatTensor(t)

ValueError: expected sequence of length 5 at dim 1 (got 6)

Apparently, it says, it is expecting the same length of lists if I am not wrong.

Can anyone help here?

tensor pytorch python
3 Answers

I agree with @helloswift123, you cannot stack tensors of different lengths.

Also, @helloswift123's answer will work only when the total number of elements is divisible by the shape that you want. In this case, the total number of elements is 19 and in no case, it can be reshaped into something useful since it is a prime number.

torch.cat() as suggested,

data = [torch.tensor([-0.1873, -0.6180, -0.3918, -0.5849, -0.3607]),
                torch.tensor([-0.6873, -0.3918, -0.5849, -0.9768, -0.7590, -0.6707]),
                torch.tensor([-0.6686, -0.7022, -0.7436, -0.8231, -0.6348, -0.4040, -0.6074, -0.6921])]
dataTensor = torch.cat(data)
dataTensor.numel()

Output:

tensor([-0.1873, -0.6180, -0.3918, -0.5849, -0.3607, -0.6873, -0.3918, -0.5849,
        -0.9768, -0.7590, -0.6707, -0.6686, -0.7022, -0.7436, -0.8231, -0.6348,
        -0.4040, -0.6074, -0.6921])
19

Possible solution:

This is also not a perfect solution but might solve this problem.

# Have a list of tensors (which can be of different lengths) 
data = [torch.tensor([-0.1873, -0.6180, -0.3918, -0.5849, -0.3607]),
        torch.tensor([-0.6873, -0.3918, -0.5849, -0.9768, -0.7590, -0.6707]),
        torch.tensor([-0.6686, -0.7022, -0.7436, -0.8231, -0.6348, -0.4040, -0.6074, -0.6921])]

# Determine maximum length
max_len = max([x.squeeze().numel() for x in data])

# pad all tensors to have same length
data = [torch.nn.functional.pad(x, pad=(0, max_len - x.numel()), mode='constant', value=0) for x in data]

# stack them
data = torch.stack(data)

print(data)
print(data.shape)

Output:

tensor([[-0.1873, -0.6180, -0.3918, -0.5849, -0.3607,  0.0000,  0.0000,  0.0000],
        [-0.6873, -0.3918, -0.5849, -0.9768, -0.7590, -0.6707,  0.0000,  0.0000],
        [-0.6686, -0.7022, -0.7436, -0.8231, -0.6348, -0.4040, -0.6074, -0.6921]])
torch.Size([3, 8])

This will append zeros to the end of any tensor which is having fewer elements and in this case you can use torch.stack() as usual.

I hope this helps!

user_118631950

Try:

>>>data = [tensor([-0.1873, -0.6180, -0.3918, -0.5849, -0.3607]),tensor([-0.6873, -0.3918, 
-0.5849, -0.9768, -0.7590, -0.6707]),tensor([-0.6686, -0.7022, -0.7436, -0.8231, 
-0.6348, -0.4040, -0.6074, -0.6921])]

>>>dataTensor = torch.cat(data).reshape(x,y)  #x*y = data.numel()

>>>print(type(dataTensor))
<class 'torch.Tensor'>

torch.stack concatenates a sequence of tensors with same size.

torch.cat concatenates a sequence of tensors.

From the documentation of torch.cat:

Concatenates the given sequence of seq tensors in the given dimension. All tensors must either have the same shape (except in the concatenating dimension) or be empty.

user_48857130

Another possible solution, use torch.nn.utils.rnn.pad_sequence

# data = [tensor([1, 2, 3]), 
#         tensor([4, 5])]
data = pad_sequence(data, batch_first=True)
# data = tensor([[1, 2, 3],
#                [4, 5, 0]])
user_82708790
Related