LSTM not able to forecast a simple time series

Question

I'm trying to test an LSTM model on the following time series:

As you can see it is stationary and periodic (not that this matters, but it should be pretty easy for a neural net to pick up). This is in fact a coordinate of a simple pendulum vs time.

The steps for preprocessing are the following:

Scale this array using MinMaxScaler.
My model will predict x[t] using x[t-1] up to x[t-5]

scaler = MinMaxScaler()
X = scaler.fit_transform(x.reshape(-1,1))
lookback = 5
features=1

model_input, labels = [],[]

for i in range(X.shape[0]-lookback):
    
    model_input.append(X[i:i+lookback])
    labels.append(X[i+lookback])


model_input = np.asarray(model_input)  
labels = np.asarray(labels)
model_input.shape, labels.shape

which returns ((495,5,1), (495,1)) this makes sense because my t has 500 steps.

Then I build and train the model:

#train on the first 400 steps, predict on the next 100
train_in, train_out = model_input[:400], labels[:400]
test_out = labels[400:]

model = Sequential()
model.add(LSTM(64, input_shape = (lookback, features))) #input shape is (batch, timesteps, features)
model.add(Dense(1))
model.compile(optimizer = 'adam', loss = 'mse')

#train
model.fit(train_in, train_out, epochs = 30)

Finally, I want to test my model. I don't see the point of using predict here. I want to use the last 5 coordinates in the training set to generate a prediction for the first step in the testing set. Then, I will use this prediction as an input to calculate the next position. And so on...

Here is the code:

#now we make predictions

preds = []

preds_input = train_in[-1:] #to make the first prediction on the test set, we start with the last batch of the training set

for i in range(test_out.shape[0]):
    
    #the next step is the prediction on preds_input
    next_step = model.predict(preds_input, verbose=0)
    
    #append next_step to preds
    preds.append(next_step)
    
    #append next_step to preds_input and remove the first value so it keeps shape 1,5,1
    preds_input = np.append(preds_input,next_step.reshape(1,1,1), axis=1)
    preds_input = preds_input[:, 1:, :,]

I then rescaled the predictions and the testing data using inverse_transform and plotted the results.

This is what I got

I'm not able to understand why my model performed so poorly. The pattern is simple and it should've been able to pick it up. Any help would be great!

LSTM not able to forecast a simple time series

0 Answers