Order of timesteps in Keras LSTM training and validation data

user_14170301 0
Viewed 509

When using Keras recurrent layers, such as an LSTM layer, your input data is supposed to be in the 3-dimensional form of [samples,timesteps,features]. The problem here is that I can't get any reliable information on the order of the timesteps dimension in the input matrices. As a concrete example, would the first element/row, for a given sample, be the most recent known row of features or the most distant row of features?

I would suspect that the order would matter since the model is recurrent, and indeed when flipping the order of the timesteps column you end of getting different training results (when you have appropriately set seeds which would yield reproducibility.)

If someone would force me to guess, I would assume that the first row (for one sample) would be the most distant features in the sequence, but is there a definite answer here?

lstm keras python
1 Answers

It's easier to picture 3D arrays in Keras as an array of 2D arrays. The first dimension is the samples.

Your array must be of the shape [samples, timesteps, features]. Picture this as an array of length samples, where each element is itself a 2D array of shape [timesteps, features]. So, to be clear, each element in this array is a 2D array with timesteps number of rows and features number of columns.

The first row (index 0) in each of these 2D arrays is the first timestep, i.e., oldest as you say. The last row in each of these 2D arrays is the last timestep, i.e., most recent. For each timestep row, you have features along the columns.

It helps to unroll the array as done above to picture it easily and with more clarity.

Hope this helped.

user_139450130
Related