Sklearn fit method error when building composed estimator

Question

I was trying to build a composed estimator in sklearn ; I have now found that sklearn.compose.TransformedTargetRegressor does exactly what I was trying to achieve, but I am still unable to replicate it and I am curious why.

The error I am getting :

AssertionError: Estimator TransformedSkModel should not change or mutate  the parameter model from LinearRegression() to LinearRegression() during fit.

My code :

import numpy as np
from sklearn.base import BaseEstimator

class TransformedSkModel(BaseEstimator):
    
    def __init__(self, model, transform_function, reverse_transform_function):
        
        # TODO: ideally, verify, based on specified ranges, that the transform
        # function and reverse transform function are compatible
        
        self.model = model
        self.transform_function = transform_function
        self.reverse_transform_function = reverse_transform_function
    
    def fit(self, X, y):
        
        # Trying to reproduce as sklearn compatible estimator, we must use the 
        # conventions:
            
        # The constructor in an estimator should only set attributes to the 
        # values the user passes as arguments. All computation should occur in 
        # fit, and if fit needs to store the result of a computation, it should 
        # do so in an attribute with a trailing underscore (_). This convention 
        # is what makes clone and meta-estimators such as GridSearchCV work.
        
        self.vectorized_transform_function_ =\
            np.vectorize(self.transform_function)
        self.vectorized_reverse_transform_function_ =\
            np.vectorize(self.reverse_transform_function)
            
        y_transformed = self.vectorized_transform_function_(y)
        self.model.fit(X, y_transformed)
        
        return self
        
    def predict(self, X):
        
        y_transformed = self.model.predict(X)
        y = self.vectorized_reverse_transform_function_(y_transformed)
        return y
    
    # def get_params(self, ):
        
    #     return self.model.get_params()
    
if __name__ == "__main__":
    
    from sklearn.utils.estimator_checks import check_estimator
    from sklearn.linear_model import LinearRegression
    
    lm = LinearRegression()
    id_func = lambda x:x
    test = TransformedSkModel(lm, id_func, id_func)
    check_estimator(test)

EDIT: I use version sklearn 0.24.0 and python version 3.6.8

user_10011330 · Answer

So from looking at sklearn.compose.TransformedTargetRegressor (which does what I want), the key seems to replicate my model using sklearn.base.clone and to fit the new self.model_ (with underscore at trail to match the convention). So the new code for my fit method becomes:

def fit(self, X, y):
        
        # Trying to reproduce as sklearn compatible estimator, we must use the 
        # conventions:
            
        # The constructor in an estimator should only set attributes to the 
        # values the user passes as arguments. All computation should occur in 
        # fit, and if fit needs to store the result of a computation, it should 
        # do so in an attribute with a trailing underscore (_). This convention 
        # is what makes clone and meta-estimators such as GridSearchCV work.
        
        self.vectorized_transform_function_ =\
            np.vectorize(self.transform_function)
        self.vectorized_reverse_transform_function_ =\
            np.vectorize(self.reverse_transform_function)
            
        y_transformed = self.vectorized_transform_function_(y)
        self.model_ = clone(self.model)
        self.model_.fit(X, y_transformed)

Now I get another error linked to using np.vectorize, but that's another issue that can be addressed in another question I guess.

Sklearn fit method error when building composed estimator

1 Answers