LSTM model fitting issue

user_19814191 0
Viewed 25

I have a very simple LSTM model defined as

def get_lstm_model(shape_input, num_output):
    model = Sequential([
        layers.Input((shape_input, num_output)),
        layers.LSTM(64),
        layers.Dense(32, activation = 'relu'),
        ])
    model.compile(optimizer='adam',
                  loss='categorical_crossentropy',
                  metrics = ['accuracy'])
    return model

The model definition works fine with the below lines

model_mlp = get_lstm_model(8,5)
model_mlp.summary()

Now when I am fitting the model, I get the error in response to this line

model_history = model_mlp.fit(x_train, y_train, validation_split=0.2,
                                  epochs=500, batch_size=5000)

The error I get is:

"Input 0 of layer "lstm_3" is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: (None, 8)"
For clarity, the shape of x_train is (2134,8), while the shape of y_train is (2134,5)

Any help would be greatly appreciated.

lstm keras tensorflow
1 Answers

You have indeed a problem inside the input dimensions. I'm not sure about what your data represents, so I've created two dummy numpy array for x_train and y_train of the desired shape.

From what I'm understanding you want an output shape of 5, so you have to specify that inside the last layer.

Regarding the input shape, the major problem is that the LSTM layer expects an input shape of three dimensions. From the documentation of LSTM:

inputs: A 3D tensor with shape [batch, timesteps, feature]

So x_train must have a 3D shape: number of samples + two numbers like I show you below.

The code:

import numpy as np
from tensorflow.keras import layers, Sequential
import tensorflow as tf


def get_lstm_model(shape_input, num_output):
  model = keras.Sequential()
  model.add(layers.LSTM(64, input_shape=(None, 8)))
  model.add(layers.Dense(32, activation = 'relu'))
  model.add(layers.Dense(5))
  model.compile(optimizer='adam',
                loss='categorical_crossentropy',
                metrics = ['accuracy'])
  return model

model_mlp = get_lstm_model(8,5)
print(model_mlp.summary())

x_train = np.zeros((2134, 8, 8))
y_train = np.zeros((2134, 5))
model_history = model_mlp.fit(x_train, y_train, validation_split=0.2, epochs=2, batch_size=5000)

The summary:

Model: "sequential_40"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 lstm_33 (LSTM)              (None, 64)                18688     
                                                                 
 dense_43 (Dense)            (None, 32)                2080      
                                                                 
 dense_44 (Dense)            (None, 5)                 165       
                                                                 
=================================================================
Total params: 20,933
Trainable params: 20,933
Non-trainable params: 0
_________________________________________________________________
user_147749590
Related