Siamese Model implementation can be found below:
class ClassifierSiameseLSTM(nn.Module):
def __init__(self, num_sensors=2, hidden_units=16):
super().__init__()
self.num_sensors = num_sensors # this is the number of features
self.hidden_units = hidden_units
self.num_layers = 1
self.lstm = nn.LSTM(
input_size=num_sensors,
hidden_size=hidden_units,
batch_first=True,
num_layers=self.num_layers)
self.fc = nn.Linear(in_features=self.hidden_units, out_features=256)
def forward_once(self, x):
batch_size = x.shape[0]
h0 = torch.zeros(
self.num_layers, batch_size, self.hidden_units,
dtype=torch.double).to(device).requires_grad_()
c0 = torch.zeros(self.num_layers, batch_size, self.hidden_units,
dtype=torch.double).to(device).requires_grad_()
output, (hn, cn) = self.lstm(x, (h0, c0))
out = self.fc(output[:, -1, :])
return out
def forward(self,x1,x2):
output1 = self.forward_once(x1)
output2 = self.forward_once(x2)
return output1,output2
When I try to train the model, I got the following loss values:
Epoch : 0
Train Loss: 0.090 | Accuracy: 64.000
Epoch : 1
Train Loss: 0.091 | Accuracy: 64.000
Epoch : 2
Train Loss: 0.090 | Accuracy: 64.000
Epoch : 3
Train Loss: 0.090 | Accuracy: 64.000
Epoch : 4
Train Loss: 0.091 | Accuracy: 64.000
Epoch : 5
Train Loss: 0.090 | Accuracy: 64.000
The loss value is almost constant and the model is not learning at all. Does anyone know the solution or have an idea about the cause of the problem?