I am trying to expand a tensor first, then assign a value to a specific index.
For example, if I have a tensor of size (1,14), I want to expand it to (1,15), then assign 0 to the beginning of the expanded tensor.
The only way I can think of doing it, is by the following code snipit.
a = torch.tensor([[1,2,3,4,5,6,7,8,9,10,11,12,13,14]]) ## torch.Size([1, 14])
b = torch.tensor([[0]]) ## torch.Size([1, 1])
x = torch.cat((a,b.expand((1,1))),dim=1) ## torch.Size([1, 15])
## answer before rolling [[ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 0]]
x = torch.roll(x, 1)
print(x.numpy()) ## [[ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14]]
However, when I copy this code to my feed forward method, it gives a size mismatch.
The reason behind adding the Zero to the first index of a tensor is that I have two modalities, e.g., image and text, and I am designing a fusion model to combine their feature vectors. However, I want to preserve the original feature vectors that come from the unimodels.
Maybe the image will make this concept clear.
This is also a code snipit from the feed forward:
class Fusion_dot_product(nn.Module):
def __init__(self, nb_classes=3):
super(Fusion_dot_product, self).__init__()
self.model_image = cnn_m
self.model_text = net_MLP
self.layer_out = nn.Linear(1024, nb_classes)
def forward(self, x1,x3):
b = torch.tensor([[0]])
x1 = self.model_image(x1) ## size of x1 is (batch_size, 32)
x1_=torch.cat((b.expand((1,1)),x1),dim=1)
x1_ = torch.roll(x1_, 1)
x3 = self.model_text(x3) ## size of x3 is (batch_size, 32)
x3 = x3.view(x3.size(0), -1)
x3_=torch.cat((b.expand((1,1)),x3),dim=1)
x3_ = torch.roll(x3_, 1)
## dot product
x =x3_.view(x3_.shape[0], x3_.shape[1], 1)+ x1_.view(x1_.shape[0], 1, x1_.shape[1])
x=x.flatten(start_dim=1)
## classifier layer
x = self.layer_out(x)
return x
Is there another way I can fix this and do it efficiently?
Any help is greatly appreciated!