I am building models for time series forecasting, by adapting the code found in Tensorflow's tutorial In particular, I want to make forecasts for the following window:
Total window size: 27
Input indices: [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25]
Label indices: [ 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26]
As you can see, the first prediction uses observations at t=0,1,2 to forecast the label at t=3; the second uses observations at t=1,2,3 to forecast the label at t=4; and so on...
One way to make these forecasts is to use a convolutional neural network such as:
conv_model = tf.keras.Sequential([tf.keras.layers.Conv1D(filters=32, kernel_size=(3,), activation='relu'),
tf.keras.layers.Dense(units=32, activation='relu'),
tf.keras.layers.Dense(units=1), ])
Once the forecasts are produced, it is possible to plot the following figure.
Is it possible to do the same type of forecast with an LSTM? In other words, can I specify a warm up period in an LSTM?
I tried to take something like this:
lstm_model = tf.keras.models.Sequential([
# Shape [batch, time, features] => [batch, time, lstm_units]
tf.keras.layers.LSTM(32, return_sequences=True),
# Shape => [batch, time, features]
tf.keras.layers.Dense(units=1) ])
and modify the return_sequence argument:
lstm_model = tf.keras.models.Sequential([
# Shape [batch, time, features] => [batch, time, lstm_units]
tf.keras.layers.LSTM(32, return_sequences=False),
# Shape => [batch, time, features]
tf.keras.layers.Dense(units=1) ])
But this breaks the dimensionality of the model.