ARIMA forecast gives different results with new python statsmodels

user_13659825 0
Viewed 4952

I'm (out-of-sample) forecasting with ARIMA(0,1,0).

In python's statsmodels latest stable version 0.12. I calculate:

import statsmodels.tsa.arima_model as stats

time_series = [2, 3.0, 5, 7, 9, 11, 13, 17, 19]
steps = 4
alpha = 0.05

model = stats.ARIMA(time_series, order=(0, 1, 0))
model_fit = model.fit(disp=0)

forecast, _, intervals = model_fit.forecast(steps=steps, exog=None, alpha=alpha)

which results in

forecast = [21.125, 23.25, 25.375, 27.5]
intervals = [[19.5950036, 22.6549964 ], [21.08625835, 25.41374165], [22.72496851, 28.02503149], [24.44000721, 30.55999279]]

and a Future Warning, which suggests:

FutureWarning: 
statsmodels.tsa.arima_model.ARMA and statsmodels.tsa.arima_model.ARIMA have
been deprecated in favor of statsmodels.tsa.arima.model.ARIMA (note the .
between arima and model) and
statsmodels.tsa.SARIMAX. These will be removed after the 0.12 release.

In the new version, as hinted to in the Future Warning, I calculate:

import statsmodels.tsa.arima.model as stats

time_series = [2, 3.0, 5, 7, 9, 11, 13, 17, 19]
steps = 4
alpha = 0.05

model = stats.ARIMA(time_series, order=(0, 1, 0))
model_fit = model.fit()

forecast = model_fit.get_forecast(steps=steps)
forecasts_and_intervals = forecast.summary_frame(alpha=alpha)

which gives different results:

forecasts_and_intervals =
y  mean   mean_se  mean_ci_lower  mean_ci_upper
0  19.0  2.263842      14.562951      23.437049
1  19.0  3.201556      12.725066      25.274934
2  19.0  3.921089      11.314806      26.685194
3  19.0  4.527684      10.125903      27.874097

I would like to obtain the same results as before. Am I using the new interface correctly?

I need both the forecast and the intervals. I tried already to use different functions as just forecast the new interface offers.

In particular I'm wondering why the forecast result is 19 for the entire list.

Many thanks for every help.

Here is the documentation for statsmodels 0.12.2: https://www.statsmodels.org/stable/generated/statsmodels.tsa.arima_model.ARIMA.html?highlight=arima#statsmodels.tsa.arima_model.ARIMA

Here is the documentation for newer version of Arima: https://www.statsmodels.org/stable/generated/statsmodels.tsa.arima.model.ARIMA.html?highlight=arima#statsmodels.tsa.arima.model.ARIMA

statsmodels arima python
2 Answers

The difference is due to whether the models include a "constant" term or not. For the first case i.e. older statsmodels.tsa.arima_model.ARIMA, it automatically includes a constant term (and no option to turn on/off). If you have a differencing, it also includes it but does so in the differenced domain (otherwise it would be eliminated anyway). So here is its ARIMA(0, 1, 0) model:

y_t - y_{t-1} = c + e_t

which is "random walk with drift".

For the new statsmodels.tsa.arima.model.ARIMA, as the documentation you linked says, not any kind of trend term (including constant, i.e. c) is included when differencing is involved, which is the case for you. So here is its ARIMA(0, 1, 0) model:

y_t - y_{t-1} = e_t

which is "random walk" and as we know, forecasts from it corresponds to naive forecasts i.e. repeating the last value (19 in your case).

Then, what to do to make the new one work?

It includes a parameter called trend which you can specify to get the same behaviour. Since you are using a differencing (d=1), passing trend="t" should give the same model as the old one. ("t" means linear trend but since d = 1, it will reduce to a constant in the differenced domain):

import statsmodels.tsa.arima.model as stats

time_series = [2, 3.0, 5, 7, 9, 11, 13, 17, 19]
steps = 4
alpha = 0.05

model = stats.ARIMA(time_series, order=(0, 1, 0), trend="t")   # only change is here!
model_fit = model.fit()

forecast = model_fit.get_forecast(steps=steps)
forecasts_and_intervals = forecast.summary_frame(alpha=alpha)

and here is what I get for forecasts_and_intervals:

y       mean   mean_se  mean_ci_lower  mean_ci_upper
0  21.124995  0.780622      19.595004      22.654986
1  23.249990  1.103966      21.086256      25.413724
2  25.374985  1.352077      22.724962      28.025008
3  27.499980  1.561244      24.439997      30.559963
user_93321870

I think this raises another issue. I'm not sure exogenous variables are treated the same in the new arima.model version. I believe in the old version, arima_model, they are applied to the order of differences. For (0,0,0) Y=mx+b or if (0,1,0), then dy=mx+b.

user_157944660
Related