Time series forecasting with ANN

Question

I am trying to forecast the power consumption for the next 10 minutes with an ANN using different inputs and parameters. The database has 26000+ samples recorded every 10 minutes. The image below is a slice of the database.

    Date_time              Consumption
0   2016-10-01 00:00:00    2781
1   2016-10-01 00:10:00    3729
2   2016-10-01 00:20:00    3357
3   2016-10-01 00:30:00    2609
4   2016-10-01 00:40:00    3025

In order to forecast "t" I used as inputs:

Last 6 periods (t-1 to t-6);
Last 4 days at same time (t-144, t-288, t-432 and t-576);
Last 3 periods and 4 days at same time;
Last 3 periods, 4 days and 4 weeks at same time (t-1008, t-2016, t-3024, t-4032).

I used the pandas shift function to create new columns with the lagged features and then dropped the NaN values. I set 20% of the data to be used as test set and then applied MinMaxScaler to transform the features.

For each input configuration I did a grid search in order to find the best parameters. The code below shows the function used to fit the models.

def model_fit(train_x, train_y, model_config):
    
    # reset keras
    
    clear_session()
    
    # model_config contains the parameters of the model
    
    n_layers, n_nodes, activation, n_batch = model_config
    
    model = Sequential()
    
    # the shape of the input layer is given by the number of features
    
    model.add(InputLayer(input_shape=(train_x.shape[1], )))
    
    # model.add(Dropout(0.2))
    
    for layers in range(n_layers):
        
        model.add(Dense(n_nodes, 
                        activation=activation
                       )
                 )
        
    model.add(Dense(1))

    model.compile(loss='mse', optimizer=Adam(0.01), metrics=['mse', 'mae'])
    
    model.summary()

    earlystop = EarlyStopping(monitor='val_mse')
    
    # 15% of the training set will be used to validate the model

    model.fit(train_x, 
              train_y, 
              validation_split=0.15, 
              callbacks = [earlystop], 
              epochs=1500, 
              batch_size=n_batch, 
              shuffle=False, 
              verbose=1
             )
    
    return model

After trying several parameters combinations, I found that configuration 3 had the best results.

MODEL CONFIGURATION: (n_layers=1, n_nodes=48, activation='relu', n_batch=50)

Training RMSE: 429.3599853515625
Validation RMSE: 223.30999755859375 

Training MAE: 271.6600036621094
Validation MAE: 159.44000244140625 

Test RMSE: 227.69
Test MAE: 167.88

The result above suggests that the ANN is just using 't-1' to forecast 't'. However, by looking at the pattern in the first figure, it seems to me that the model can do better than this. I found some similar questions and I have already tried some of the suggestions. When using dropout, the shifted/delayed forecast problem seems to disappear (figure below). I also tried to use the time (transformed with cosine and sine to be cyclical) as input. However, the results are much worse and now I don't really know what else to do.

===============================================================

Before working with the whole database, I tried to develop a model with a smaller database which had the measurements hourly and it worked pretty well. The inputs were the time (transformed with cosine and sine), consumption at the last period (t-1) and at the last day at the same time (t-24). The figure below shows the result.

MODEL CONFIGURATION: (n_layers=1, n_nodes=48, activation='tanh', n_batch=50)
MODEL RMSE: 354.76
MODEL MAE: 283.97

Time series forecasting with ANN

0 Answers