I'm trying to pass a parameter into a Transformer (BaseEstimator, TransformerMixin) wrapped in a pipeline using set_params, but it's not working, even though get_params on the pipe containing the transformer shows the parameter is there.
It needs to be passed in, and not sourced from globals/locals or in the same module. Most examples I've seen either use the later approach or hardcode as an argument inside the pipe definition, but the documentation on set_params makes it pretty clear it should work when passing into a transformer in a pipe (or even nested pipes.) Just not so clear how to actually do this.
Rather than going on about it, here's a complete toy example for the code to speak for itself:
import pandas as pd
from sklearn.pipeline import Pipeline
from sklearn.base import BaseEstimator, TransformerMixin
class SomeTransform(BaseEstimator, TransformerMixin):
def __init__(self, param):
self.param = param
def fit(self, X, y=None):
return self
def transform(self, X):
# do something with param
print("here's our param: ", self.param)
return X
some_pipe = Pipeline([
#('st', SomeTransform(param='foo')), # It works if param is hardcoded in pipe.
('st', SomeTransform()), # It doesn't work if param is passed to pipe.
])
some_pipe[0].set_params(param='bar')
print(some_pipe[0]) # SomeTransform(param='bar')
print(some_pipe[0].get_params()) # {'param': 'bar'}
# data doesn't matter for this toy example.
data = [['a', 1], ['b', 2], ['c', 3]]
df = pd.DataFrame(data, columns=['letters', 'numbers'])
wat = some_pipe.transform(df)
As commented, it works fine if the param is passed from inside the pipe itself, but doesn't work when the param is passed to the pipe.
How do I pass this param so it is available to the transformer? The only examples I have found all use GridSearchCV, which is not what I am using. The error I'm getting is similar to this scikit bug, but I'm only passing a single param, not multiple.
I rarely "give up" and ask SO, but really hitting a wall here, and pretty sure I'm just missing the obvious.