What is the timestep in Keras' LSTM?

user_4877104 0
Viewed 9029

I have some troubles with the LSTM implementation in Keras.

My training set is structured as follow:

  • number of sequences: 5358
  • the length of each sequence is 300
  • each element of the sequence is a vector of 54 features

I'm unsure on how to shape the input for a stateful LSTM.

Following this tutorial: http://philipperemy.github.io/keras-stateful-lstm/, I've created the subsequences (in my case there are 1452018 subsequences with a window_size = 30).

What is the best option to reshape the data for a stateful LSTM's input?

What means the timestep of the input in this case? And why?

Is the batch_size related to the timestep?

lstm keras tensorflow python
1 Answers

I'm unsure on how to shape the input for a stateful LSTM.

LSTM(100, statefull=True)

But before using stateful LSTM ask yourself do I really need statefull LSTM? See here and here for more details.

What is the best option to reshape the data for a stateful LSTM's input?

It really depends on the problem on hands. However, I think you do not need reshaping just feed data directly into Keras:

input_layer = Input(shape=(300, 54))

What means the timestep of the input in this case? And why?

In your example timestamp is 300. See here for further details on timestamp. In the following picture, we have 5 timestamps that we feed them into the LSTM network.

enter image description here

Is the batch_size related to the timestep?

No, it has nothing to do with batch_size. More details on batch_size can be found here.

Here is simple code based on the description that you provide. It might give you some intuition:

import numpy as np
from tensorflow.python.keras import Input, Model
from tensorflow.python.keras.layers import LSTM
from tensorflow.python.layers.core import Dense

x_train = np.zeros(shape=(5358, 300, 54))
y_train = np.zeros(shape=(5358, 1))

input_layer = Input(shape=(300, 54))
lstm = LSTM(100)(input_layer)
dense1 = Dense(20, activation='relu')(lstm)
dense2 = Dense(1, activation='sigmoid')(dense1)

model = Model(inputs=input_layer, ouputs=dense2)
model.compile("adam", loss='binary_crossentropy')
model.fit(x_train, y_train, batch_size=512)
user_14627700
Related