Partial fit or incremental learning for autoregressive model

user_7978198 0
Viewed 404

I have two time series representing two independent periods of data observation. I would like to fit an autoregressive model to this data. In other words, I would like to perform two partial fits, or two sessions of incremental learning.

This is a simplified description of a not-unusual scenario which could also apply to batch fitting on large datasets.

How do I do this (in statsmodels or otherwise)? Bonus points if the solution can generalise to other time-series models like ARIMA.

In pseudocode, something like:

import statsmodels.api as sm
from statsmodels.tsa.ar_model import AutoReg

data = sm.datasets.sunspots.load_pandas().data['SUNACTIVITY']
data_1 = data[:len(data)//3]
data_2 = data[len(data)-len(data)//3:]

# This is the standard single fit usage
res = AutoReg(data_1, lags=12).fit()
res.aic

# This is more like what I would like to do
model = AutoReg(lags=12)
model.partial_fit(data_1)
model.partial_fit(data_2)
model.results.aic
autoregressive-models statsmodels time-series machine-learning python
2 Answers

Statsmodels does not directly have this functionality. As Kevin S mentioned though, pmdarima does have a wrapper that provides this functionality. Specifically the update method. Per their documentation: "Update the model fit with additional observed endog/exog values.".

See example below around your particular code:

from pmdarima.arima import ARIMA
import statsmodels.api as sm

data = sm.datasets.sunspots.load_pandas().data['SUNACTIVITY']
data_1 = data[:len(data)//3]
data_2 = data[len(data)-len(data)//3:]

# This is the standard single fit usage
model = ARIMA(order=(12,0,0))
model.fit(data_1)

# update the model parameters with the new parameters
model.update(data_2)
user_168237750

I don't know how to achieve that in autoreg, but I think it can be achieved somehow, but need to manually evaluate results or somehow add the data.

But in ARIMA and SARIMAX, it's already implemented and it's simple.

For incremental learning, there are three functions related and it's documented here. First is apply which use fitted parameters on new unrelated data. Then there are extend and append. Append can be refit. I don't know exact difference though.

Here is my example that is different but similar...

from statsmodels.tsa.api import ARIMA

data = np.array(range(200))

order = (4, 2, 1)

model = ARIMA(data, order=order)
fitted_model = model.fit()


prediction = fitted_model.forecast(7)

new_data = np.array(range(600, 800))
fitted_model = fitted_model.apply(new_data)
new_prediction = fitted_model.forecast(7)

print(prediction)  # [200. 201. 202. 203. 204. 205. 206.]
print(new_prediction)  # [800. 801. 802. 803. 804. 805. 806.]

This replace all the data, so it can be used on unrelated data (unknown index). I profiled it and apply is very fast in comparison to fit.

user_102793360
Related