Smooth LSTM predictions

Viewed 768

I am new to deep learning models. This is my first post, so please do let me know if there are any additional details that you may need.

Here is a general summary of my approach:

  • I used a dataset having two time series for training
  • I converted them to the dataset accepted by keras' LSTM/Bi-LSTM layers in the format:
    [1, 0.99, 0.98, 0.97] ==Output==> [0.96]
    and so on..
  • Shapes of the input and output containers (arrays): input(794, 5, 1) and output(794, )
  • Error-free training
  • Prediction on an initial points of data (window) having shape (1, 5, 1)
  • The predicted output is a value, which is appended to a separate list (for plotting), as well as appended to the window, and the first value of the window dropped out. This window is then fed as input to the model to generate the next prediction point.
  • Continue this until I get the whole curve for both models (LSTM and Bi-LSTM)

However, the prediction is not even close to the actual data.

Obtained curves

With some changes in the batch sizes and other parameters, I am still able to get the TREND (downwards in this case), but not the SHAPE of the data (edges, in-between variations, absolutely nothing).

Is it because of using only two time series for training? I cannot exactly figure out the problem.
Model (similar code goes for Bi-LSTM model):

model_lstm = Sequential()
model_lstm.add(LSTM(128, input_shape=(timesteps, 1), return_sequences= True))
model_lstm.add(Dropout(0.1))
model_lstm.add(LSTM(128, return_sequences= True))
model_lstm.add(Dropout(0.1))
model_lstm.add(LSTM(128, return_sequences= False))
model_lstm.add(Dropout(0.1))
model_lstm.add(Dense(1))
model_lstm.compile(loss = 'mean_squared_error', optimizer = optimizers.Adam(0.001))

Curve prediction code:
timesteps = 5
start = cell_to_test[0:timesteps].reshape(1, timesteps, 1)   # Has the initial data points
yhat = 1.0
y_curve_lstm = list(start.flatten())                         # The list for storing the curve points
y_window = start                                             # Window


The loop to predict the individual points until a limit is reached: (The limits of the cell data whose curve I am predicting):

while len(y_curve_lstm) <= len(cell_to_test):
  yhat = model_lstm.predict(y_window)
  yhat = float(yhat)
  y_curve_lstm.append(yhat)
  y_window = list(y_window.flatten())
  y_window.append(yhat)
  y_window.remove(y_window[0])
  y_window = np.array(y_window).reshape(1, timesteps, 1)
  #print(yhat)
0 Answers
Related