In Keras, after you train a stateful LSTM model, do you have to re-train the model as you predict values?

user_13226563 0
Viewed 308

I've been trying to create a stateful LSTM model with keras, and I pretty much figured out the training part, but I don't get the predicting part.

So, let's imagine that we had 10000 time-series datapoints. we use 9000 in front for training, and the other 1000 for testing. So, as we start training, we set the window length to 2, and slide the window forward as we set the input(X) as the first datapoint and set the output(y) as the second datapoint. X as input and y as output

And as we train, the model converges because of it's stateful nature. Finally we finish training.

Now, we are left with a model, and some test data. The problem begins here. We test the first datapoint. enter image description here

It returns a guessed value. Nice.

We test the second datapoint of the test set. enter image description here

We get an output. But, the problem is that because we were using a stateful model, and we only one value as an input, the only way the model is going to figure out the next value is from memory of the previous time-series.

But since we didn't train the data on the first datapoint of the test set, the time-series is broken, and the model will think that the second datapoint on the test set is the first datapoint on the test set!

So, my question is,

  1. does keras take care of this and automaticaly train the network as it's predicting?
  2. or do I have to train the net as I am predicting
  3. or is there some other reason that enables me to just keep predicting without training the model farther?
keras tensorflow deep-learning machine-learning python
1 Answers

For a stateful LSTM, if will retain information in its cells as you predict. If you were to take any random point in the train or test dataset and repeatedly predict on it, your answer will change each time, because it keeps seeing this data and uses it every time it predicts. The only way to get a repeatable answer would be to call reset_states().

You should be calling reset_states() after each training epoch, and when you save the model, those cells should be empty. Then if you want to start predicting on the test set, you can predict on the last n training points (without saving the values anywhere), then start saving values once you get to your first test point.

It is often good practice to seed the model before prediction. If I want to evaluate on test_set[10:20,:], I can let the model predict on test_set[:10,:] first to seed the model then start saving my predicted values once I get to the range I am interested in.

To address the further training question, you do not need to train the model further to predict. Training will only be for tuning the model's weights. Look into this blog for more information on Stateful vs Stateless LSTM.

user_132961380
Related