Keras error Incompatible shapes: [32,168,24] vs. [32,24]

user_5838180 0
Viewed 1336

I am training neural net for time series predictions. My data looks like this:

train_generator = TimeseriesGenerator(X, y, length=7*24, stride=24, batch_size=32)
print(X.shape, y.shape)
>>>((126336, 3), (126336, 24))

The training works fine on this architecture:

model = Sequential()
model.add(LSTM(16, return_sequences=True, input_shape=(7*24, X.shape[1]))) 
model.add(Dropout(0.3))
model.add(LSTM(32, return_sequences=True, input_shape=(7*24, X.shape[1])))
model.add(Dropout(0.3))
model.add(LSTM(64))
model.add(Dense(24))

model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mse'])
model.fit(train_generator, validation_data=val_generator, epochs=15, verbose=1)

However, I tried to train on this simpler architecture:

model = Sequential()
model.add(LSTM(32, return_sequences=True, input_shape=(7*24, X.shape[1])))
model.add(Dense(24))

In the second case I get an error message:

InvalidArgumentError:  Incompatible shapes: [32,168,24] vs. [32,24]

What am I doing wrong? How can I make training possible on the second architecture? And is there a general rule I have to consider so that there are no shape incompatibilities? Tnx

keras tensorflow reshape neural-network python
1 Answers

It is because you have return_sequences set to True, and you are sending that directly to a dense layer. In the original model, you had 2 LSTM layers returning sequences (which is required for LSTM -> LSTM), but your last LSTM layer did not return sequences to your output layer. That's why it worked.

Just set that to False in your simple model, and it will work. You can also get rid of this argument, because the default is False.

To elaborate on return_sequences, in LSTM layers you use information from multiple time steps to make a prediction. That is why the input shape for an LSTM layer requires information in a time-series format, not just the batch size and number of features. To pass the output of one LSTM to another, you need to provide the layer's output for all of these time steps, so you get an output shape that has a dimension of size 32x168x24 in your case.

If you want the LSTM output to be sent to a dense layer, you only want to pass the information about the current point in time, so your output should be size batch_size x n_features instead of batch_size x n_time_steps x n_features.

user_132961380
Related