Why does LSTM model produce different predictions across multiple model runs?

Viewed 306

I am using long short term memory (LSTM) to generate predictions. I have noticed that each time I run the LSTM model, it generates slightly different predictions with the same data. I was wondering why this happens and if there is something I am doing wrong?

Thank You

from numpy import array
from keras.models import Sequential
from keras.layers import LSTM
from keras.layers import Dense
from keras.layers import Flatten
from keras.layers import TimeDistributed
from keras.layers.convolutional import Conv1D
from keras.layers.convolutional import MaxPooling1D

# split a univariate sequence into samples
def split_sequence(sequence, n_steps):
    X, y = list(), list()
    for i in range(len(sequence)):
        # find the end of this pattern
        end_ix = i + n_steps
        # check if we are beyond the sequence
        if end_ix > len(sequence)-1:
            break
        # gather input and output parts of the pattern
        seq_x, seq_y = sequence[i:end_ix], sequence[end_ix]
        X.append(seq_x)
        y.append(seq_y)
    return array(X), array(y)

def LSTM_Model(Data, N_Steps, Epochs):
    # define input sequence
    raw_seq = Data

    # choose a number of time steps
    n_steps_og = N_Steps

    # split into samples
    X, y = split_sequence(raw_seq, n_steps_og)

    # reshape from [samples, timesteps] into [samples, subsequences, timesteps, features]
    n_features = 1
    n_seq = 2
    n_steps = 2
    X = X.reshape((X.shape[0], n_seq, n_steps, n_features))


    # define model
    model = Sequential()
    model.add(TimeDistributed(Conv1D(filters=64, kernel_size=1, activation='relu'), input_shape=(None, n_steps, n_features)))
    model.add(TimeDistributed(MaxPooling1D(pool_size=2)))
    model.add(TimeDistributed(Flatten()))
    model.add(LSTM(50, activation='relu'))
    model.add(Dense(1))
    model.compile(optimizer='adam', loss='mse')


   # fit model
   model.fit(X, y, epochs=Epochs, verbose=2)
   #Create Forcasting data
   #Now take the last 4 days of the Model data for the forcast
   Forcast_data = Data[len(new_data) - n_steps_og:]

   # demonstrate prediction
   x_input = array(Forcast_data)
   x_input = x_input.reshape((1, n_seq, n_steps, n_features))
   yhat = float(model.predict(x_input, verbose=0))
   return(yhat)
1 Answers

Many methods like this are initialized with random weights for the coefficients. Then they search for a good local minimum to some sort of loss function. This means they will (hopefully) find just one of the many nearly optimal solutions, but are unlikely to find the single very best solution, nor to even find the same solution repeatedly. Due to this, your results are typical, so long as your predictions are only slightly different.

This is more of a general machine learning question, rather than being specific to Python, but I hope this helps.

Related