I would like to calculate RMSE and MSE for Prophet , SARIMA and Ensemble models from KATS

Viewed 389

I have conducted an analysis where I have forecasted my data frame by converting it into timeseries data. I would like to calculate RMSE and MSE for Prophet, SARIMA and Ensemble models from KATS.

For SARIMA Code:

from kats.models.sarima import SARIMAModel, SARIMAParams
warnings.simplefilter(action='ignore')
# create SARIMA param class
params = SARIMAParams(p=2 , 
    d=1, 
    q=1, 
    trend = 'ct', 
    seasonal_order=(1,0,1,12)
    )
# initiate SARIMA model
m = SARIMAModel(data=ts, params=params)
# fit SARIMA model
m.fit()

# generate forecast values
fcst = m.predict(
    steps=HOURS ,
    include_history = True
    )

# make plot to visualize
plt1 = m.plot()
plt.xlabel('Time (in days)')
plt.ylabel('Noise Level (db)')

Forecasted result: SARIMA Forecasting

For Prophet

Code:

from kats.models.prophet import ProphetModel, ProphetParams

# create a model param instance
params = ProphetParams(seasonality_mode='multiplicative') # additive mode gives worse results

# create a prophet model instance
m = ProphetModel(ts, params)

# fit model simply by calling m.fit()
m.fit()

# make prediction for next HOURS hours
fcst = m.predict(steps=HOURS, include_history = True)

# plot to visualize
plt2 = m.plot()
plt.xlabel('Time (in days)')
plt.ylabel('Noise Level (db)')

Forecasted result:

Prophet Forecasting

Can anyone please tell me how can I achieve it? I have tried several summary statistic techniques but it doesn't seem to work. Thanks!

1 Answers

The Kats 201 tutorial notebook has examples in sections 4 and 5 showing how to use Kats to compute an evaluation_function (here, MAE) and to perform backtesting to compute many error metrics (MAPE, SMAPE, MAE, MASE, MSE, RMSE).

Related