set_params() in sklean pipeline not working with TransformTargetRegressor

user_3877781 0
Viewed 96

I would like to make a prediction of a single tree of my random forest. However, if I wrap my pipeline around TransformedTargetRegressor .set_params does not seem to work.

Please find below an example:

from sklearn.datasets import load_boston
from sklearn.compose import TransformedTargetRegressor
from sklearn.pipeline import Pipeline
from sklearn.ensemble import RandomForestRegressor
from sklearn.preprocessing import StandardScaler

# loading data
boston = load_boston()
X = boston["data"]
Y = boston["target"]

# pipeline and training
pipe = Pipeline([      
                    ('scaler', StandardScaler()),
                    ('model', RandomForestRegressor(n_estimators = 100, max_depth = 4, random_state = 0))
                 ])
treg = TransformedTargetRegressor(regressor=pipe, transformer=StandardScaler())
treg.fit(X, Y)

# single tree from random forest
tree = treg.regressor_.named_steps['model'].estimators_[0]


x_sample = X[0:1]
print('baseline: ', treg.predict(x_sample))

x_scaled = treg.regressor_.named_steps['scaler'].transform(x_sample)
y_predicted = tree.predict(x_scaled)
y_transformed = treg.transformer_.inverse_transform([y_predicted])
print("internal pipeline changes: ", y_transformed)

new_model = treg.set_params(**{'regressor__model': tree})
y_predicted = new_model.predict(x_sample)
print('with set_params(): ', y_predicted)

The output that I am getting is shown below. I would expect 'with set_params()' to be the same like 'internal pipeline changes:

baseline: [26.41013313]

internal pipeline changes: [[30.02424242]]

with set_params(): [26.41013313]

scikit-learn pipeline random-forest python
2 Answers

Apparently, scikit-learn TransformedTargetRegressor objects don't allow you to change the regressor used to predict, unless you re-fit the dataset on the new regressor in set_params. If you do this:

new_model = treg.set_params(**{'regressor__model': tree})
print(new_model)

you can see that the new parameters have been set. However, as you correctly discovered, the estimator used in predict is still the old one. If you want to change the estimator in the object, you can do:

new_model = treg.set_params(**{'regressor__model': tree})
new_model.fit(X, Y)

new_model.predict(x_sample)

And you can see that the prediction changes and uses the single tree to perform the estimation. If you are interested in the sinlge tree's prediciton and not re-fit on the whole dataset, you can just call, tree.predict() separately.

user_37916400

TransformedTargetRegressor has a parameter regressor and an attribute regressor_. The former can be set with set_params and is considered a hyperparameter, but is not used in prediction; rather, it is cloned and fitted when the TTR is fitted, and stored in the regressor_ attribute.

So you cannot use set_params to update the fitted regressor attribute. (You can check that in your code, new_model.regressor_['model'] is still a random forest.) The best you can do is directly modify the attribute (though this is probably unorthodox, and in some situations may lead to other issues):

import copy
mod_model = copy.deepcopy(treg)
mod_model.regressor_.steps[-1] = ('model', tree)
y_predicted = mod_model.predict(x_sample)
print('with modifying regressor: ', y_predicted)
user_104958930
Related