Passing parameters to a sklearn custom transformer pipeline

Viewed 401

I am trying to use a custom function in Function Transformer in Python and building a pipeline with it. The custom function takes a data frame as input and returns another data frame which is result of a group by operation. (My actual function is quite complex...using a smaller function for demo purposes). The custom function seems to work fine but itself but doesn't work when passed into a pipeline.

Custom function is given below:

mydf = pd.DataFrame({'classLabel':[0,0,0,1,1,0,0,0],
                   'categorical':[7,8,9,5,7,5,6,4],
                   'numeric1':[7,8,9,5,7,5,6,4],
                   'numeric2':[7,8,9,5,7,5,6,8]})

from sklearn.base import BaseEstimator, TransformerMixin
class summary_data1(BaseEstimator, TransformerMixin):
    """Concat the 'title', 'body' and 'code' from the results of 
    Stackoverflow query
    Keys are 'title', 'body' and 'code'.


    """

    def __init__(self, label_column):

        self.label_column = label_column

    def fit(self, x, y=None):
        return self

    def transform(self, x):
        x = x.groupby(self.label_column).sum()
        return x

summ = summary_data1(label_column='classLabel')
summ.fit(mydf)
summ.transform(mydf)

This generates output as expected:

categorical numeric1    numeric2
classLabel          
        0   39  39  43
        1   12  12  12

However, the same function doesn't seem to work with a pipeline.

from sklearn.pipeline import make_pipeline, Pipeline
from sklearn.preprocessing import FunctionTransformer

preprocessor = make_pipeline(FunctionTransformer(summary_data1))

preprocessor.fit(mydf,label_column='classLabel')

This throws the following error:

ValueError: Pipeline.fit does not accept the label_column parameter. You can pass parameters to specific steps of your pipeline using the stepname__parameter format, e.g. `Pipeline.fit(X, y, logisticregression__sample_weight=sample_weight)`.

How do I pass an argument to the function using a pipeline? Any help would be appreciated. Thank you.

0 Answers
Related