Time series prediction using LSTM - batch train and real-time prediction

Question

I am working on a path prediction problem where I am predicting the path (Latitude, Longitude) one time step ahead. I have the path data for nearly 1500 "events", which I use for training the LSTM model. For training, since I know the path a priori, I shift the time series by one step, and use it as a target vector. For example: Event 1:

Lat (t), Lon (t) --> Lat (t+1), Lon (t+1)

Lat (t+1), Lon (t+1) --> Lat (t+2), Lon (t+2)

However, for testing since the path is not known a priori, I take the trained LSTM model, and predict one time step at a time, and feed the predicted value as an input for the next time step. Below are the snippets from my code-

# Extract only the Lat, Lon values to arrays
train_full = train_df[['LatNor','LonNor','LatLag1Nor','LonLag1Nor']].values
test_full = test_df[['LatNor','LonNor','LatLag1Nor','LonLag1Nor']].values
print('train_full.shape = ', train_full.shape)
print('test_full.shape = ', test_full.shape)

# Separate the Inputs and Targets
x_train_full = train_full[:,0:2]
y_train_full = train_full[:,2:4]
x_test_full = test_full[:,0:2]
y_test_full = test_full[:,2:4]


# Defining the LSTM model
model = Sequential()
model.add(LSTM(40,input_shape=(None,2), return_sequences=True))
model.add(Dropout(0.1))
model.add(LSTM(20,input_shape=(None,2), return_sequences=True))
model.add(Dropout(0.1))
model.add(Dense(2))
model.add(Activation('linear'))
model.compile(loss='mean_squared_error', optimizer='rmsprop', metrics=['accuracy'])
model.summary()

epochs = 50
for i in range(epochs):
    print ('Running Epoch No: ', i)
    for stormID, data in train_df.groupby('EventID'):
        train = data[['LatNor','LonNor','LatLag1Nor','LonLag1Nor']]
        train = train.values
        x_train = np.expand_dims(train[:,0:2], axis=0)
        y_train = np.expand_dims(train[:,2:4], axis=0)
        #print (x_train.shape, y_train.shape)
        model.train_on_batch(x_train,y_train)
    model.reset_states()

print('Model training done.....')

#Use the optimized weights to estimate target values for training data
train_pred = new_model.predict_on_batch(np.expand_dims(train_df[['LatNor','LonNor']].values, axis=0))
train_pred_val = x_scaler.inverse_transform(train_pred[0])

The model trains well (see plots below)

enter image description here

When I use the trained model, and do a predict_on_batch on the test data, it works great. But, in reality we would not know the time series ahead of time. So, when I predict one instance at a time for the test set, and use it as an input to for the next time step, it is not working well. I suspect I am missing something, and changing the state/weights of the trained network whenever I make a predict call to the network.

    x_values = TestDF[['LatNor','LonNor']].values
    x_values_scaled = x_values
    start = x_values_scaled[0,:]
    startX = start.reshape(1,1,2)
    Results = np.empty(())
    Results = x_scaler.inverse_transform(startX[0])
    for i in range(x_values.shape[0]):
        nextLoc = model.predict(startX)
        nextLoc_rescaled = x_scaler.inverse_transform(nextLoc[0])
        Results = np.vstack((Results,nextLoc_rescaled))
        startX = nextLoc

Any thoughts or recommendations?

Time series prediction using LSTM - batch train and real-time prediction

0 Answers