I am trying to run a vector autoregression on a large list of dataframes of mortgage prepayment rates over time clustered by different cohorts (bond security, coupon, and vintage). I am running a test of this method on just one of the dataframes (below), we'll call it scaled_data and it's the same as dfs[4:5] in the for loop below.
And here is the code I'm using:
columns = scaled_data.columns.tolist()
cols = [c for c in columns if c not in ['Age_ATI', 'Vintage', 'owam','ScheduledBalance', 'SMM', 'SRCDate', 'cluster', 'PredictionDate', 'CprTarget', 'bondsec_code', 'Coupon']]
i = 1
for df in dfs[4:5]:
if df.empty is False:
df = df.loc[:, (df != df.iloc[0]).any()]
df['y_m'] = df['SRCDate'].dt.to_period('M')
df.index = df['y_m']
df.drop(columns = ['owam'], axis = 1, inplace = True)
=
train = df[df['SRCDate'] <= max(df['SRCDate']) - relativedelta(months = 3)]
test = df[df['SRCDate'] > max(df['SRCDate']) - relativedelta(months = 3)]
if train.empty is False:
X_train = train[cols]
y_train = train['CprTarget']
X_test = test[cols]
y_test = test['CprTarget']
train_size=int(len(X_train))
test_size = int(len(y_test))
model = VAR(X_train)
model_fitted = model.fit(1)
lag_order = model_fitted.k_ar
# Input data for forecasting
forecast_input = df.values[-lag_order:]
nobs = 1
fc = model_fitted.forecast(y=forecast_input, steps=nobs)
df_forecast = pd.DataFrame(fc, index=df.index[-nobs:], columns=df.columns + '_2d')
I want to plot the forecast v. actuals to assess the viability and accuracy of the model, but when I try to run the code at the bottom of the loop I get this traceback:
ValueError Traceback (most recent call last)
Input In [158], in <cell line: 8>()
45 forecast_input = df.values[-lag_order:]
46 nobs = 1
---> 47 fc = model_fitted.forecast(y=forecast_input, steps=nobs)
48 df_forecast = pd.DataFrame(fc, index=df.index[-nobs:], columns=df.columns + '_2d')
49 print(df_forecast)
File ~\Anaconda3\lib\site-packages\statsmodels\tsa\vector_ar\var_model.py:1163, in VARProcess.forecast(self, y, steps, exog_future)
1161 else:
1162 exog_future = np.column_stack(exogs)
-> 1163 return forecast(y, self.coefs, trend_coefs, steps, exog_future)
File ~\Anaconda3\lib\site-packages\statsmodels\tsa\vector_ar\var_model.py:262, in forecast(y, coefs, trend_coefs, steps, exog)
259 prior_y = forcs[h - i - 1]
261 # i=1 is coefs[0]
--> 262 f = f + np.dot(coefs[i - 1], prior_y)
264 forcs[h - 1] = f
266 return forcs
File <__array_function__ internals>:5, in dot(*args, **kwargs)
ValueError: shapes (10,10) and (16,) not aligned: 10 (dim 1) != 16 (dim 0)
Any ideas why? Alternatively, are there other methods I can use to assess the model accuracy? I'm guessing my dataset (or rather, each dataframe) is just too small to be useful for running a time-series on it.
