I want a stacked regression where the final_estimator is regfinal and the estimators are reg1 and reg2. For example:
reg1 = RandomForestRegressor()
reg2 = KNeighborsRegressor()
regfinal = ridge_regression()
I am unsure about the parametrization of the estimators so I want to perform a grid search first.
I see two ways.
The first way gives model_A, which is StackingRegressor with two GridSearchCV (on reg1 and reg2) as estimators:
params_reg1 = {some grid}
params_reg2 = {some grid}
grid1 = GridSearchCV(estimator=reg1, param_grid=params_reg1)
grid2 = GridSearchCV(estimator=reg2, param_grid=params_reg2)
estimators = [
('rfr', grid1),
('knr', grid2)
]
model_A = StackingRegressor(
estimators=estimators,
final_estimator=regfinal)
The second way gives model_B which is a GridSearchCV on a StackingRegressor model with reg1 and reg2 as estimators:
estimators = [
('rfr', reg1),
('knr', reg2)
]
params_reg_all = {some grid for reg1 and reg2}
stacked = StackingRegressor(
estimators=estimators,
final_estimator=regfinal)
model_B = GridSearchCV(estimator=stacked, param_grid=params_reg_all)
What are the differences between model_A and model_B?