Multioutput Stacking Regressor

Viewed 2334

One-shot question: I'm trying to build a Multiputput Stacked Regressor (added to sklearn 0.22). As far I understand, I have to combine StackingRegressor and MultiOutputRegressor. After several attemps this seems to be the right order:

import numpy as np 
from sklearn.ensemble import RandomForestRegressor
from sklearn.linear_model import OrthogonalMatchingPursuit
from sklearn.ensemble import StackingRegressor
from sklearn.neighbors import KNeighborsRegressor
from sklearn.multioutput import MultiOutputRegressor
from sklearn.svm import SVR

estimators = [  ('svr', SVR(kernel='rbf', C=1e3, gamma=0.1)),  
        ('knn',KNeighborsRegressor(n_neighbors=5)),
        ('omp', OrthogonalMatchingPursuit())
         ]

reg = MultiOutputRegressor(StackingRegressor( estimators = estimators, final_estimator= RandomForestRegressor(n_estimators=5)))
X=np.random.random((200,20))
y = np.random.random((200,4))
reg.fit(X,y)

reg.predict(X)

But the predict method ends with an error

*** ValueError: The base estimator should implement a predict method

I searched such error in the sklean files and it seems related to the MultiOutputRegressor:

if not hasattr(self.estimator, "predict"):
    raise ValueError("The base estimator should implement a predict method")

So i tried to look at the self.estimator model:

reg.estimator.predict(X)

but I obtain this error:

*** AttributeError: 'StackingRegressor' object has no attribute 'final_estimator_'

Looking at the attributes of reg.estimator I can not find final_estimator_ but only final_estimator so my solution is to create such attribute:

reg.estimator.final_estimator_ = reg.estimator.final_estimator

It works but I'm not sure anymore if my model now is doing what it is suppose to do (maybe it is using the same final estimator for each coordinate of the output). Is this a bug due to the combination StackingRegressor + MultiOutputRegressor or I'm missing something?

Thanks!

1 Answers

Set stacking_method='predict' in Model Initialization, and it should work fine. Idk why the 'auto' option doesn't work, but quick fix like so:

model = StackingClassifier(estimators=level0, final_estimator=level1, stack_method='predict', cv=5)
Related