How to pass parameters from Pipeline object to Transformer (scikit-learn)

Viewed 537

I'm trying to pass a parameter into a Transformer (BaseEstimator, TransformerMixin) wrapped in a pipeline using set_params, but it's not working, even though get_params on the pipe containing the transformer shows the parameter is there.

It needs to be passed in, and not sourced from globals/locals or in the same module. Most examples I've seen either use the later approach or hardcode as an argument inside the pipe definition, but the documentation on set_params makes it pretty clear it should work when passing into a transformer in a pipe (or even nested pipes.) Just not so clear how to actually do this.

Rather than going on about it, here's a complete toy example for the code to speak for itself:

import pandas as pd
from sklearn.pipeline import Pipeline
from sklearn.base import BaseEstimator, TransformerMixin

class SomeTransform(BaseEstimator, TransformerMixin):
    def __init__(self, param):
        self.param = param

    def fit(self, X, y=None):
        return self

    def transform(self, X):
        # do something with param
        print("here's our param: ", self.param)
        return X

some_pipe = Pipeline([
    #('st', SomeTransform(param='foo')),   # It works if param is hardcoded in pipe.
    ('st', SomeTransform()),                # It doesn't work if param is passed to pipe.
    ])

some_pipe[0].set_params(param='bar')
print(some_pipe[0])     #  SomeTransform(param='bar')
print(some_pipe[0].get_params())    # {'param': 'bar'}

# data doesn't matter for this toy example.
data =  [['a', 1], ['b', 2], ['c', 3]]
df = pd.DataFrame(data, columns=['letters', 'numbers'])

wat = some_pipe.transform(df) 

As commented, it works fine if the param is passed from inside the pipe itself, but doesn't work when the param is passed to the pipe.

How do I pass this param so it is available to the transformer? The only examples I have found all use GridSearchCV, which is not what I am using. The error I'm getting is similar to this scikit bug, but I'm only passing a single param, not multiple.

I rarely "give up" and ask SO, but really hitting a wall here, and pretty sure I'm just missing the obvious.

0 Answers
Related