GridSearchCV: Can't access classifier parameter of feature selection pipeline from GridSearchCV param_grid

useryMFA 0
Viewed 18

I am using SelectFromModel for feature selection and LogisticRegression as estimator. I have a preprocessing pipeline for tuning numerical and categorical columns. And using this feature selection pipe with a model in a pipeline in GridSearchCV.

In the param_grid I want to access the max_iter of LogisticRegression in the SelectFromModel method. So I tried 'selectfrommodel__logisticregression__max_iter': [400, 500],.

preprocessor = make_column_transformer(
    (num_transformer, make_column_selector(dtype_include=np.number)),
    (cat_transformer, make_column_selector(dtype_include=object))
)

fs_pipe = make_pipeline(
        preprocessor,
        SelectFromModel(estimator=LogisticRegression(solver='saga'))
)

lr_pipe = make_pipeline(fs_pipe, LogisticRegression(n_jobs=-1))

param_grid = {
    'selectfrommodel__logisticregression__max_iter': [400, 500],
    'logisticregression__penalty': ['l1', 'l2'],
    'logisticregression__solver': ['saga'],
    'logisticregression__max_iter': [400, 500],
}

lr_grid = GridSearchCV(
    estimator=lr_pipe,
    param_grid=param_grid,
    verbose=1, scoring='f1_micro',
    error_score='raise')

lr_grid.fit(trainX, trainY)

But it's throwing this error:

ValueError: Invalid parameter selectfrommodel for estimator Pipeline(steps=[('pipeline',
                 Pipeline(steps=[('columntransformer',
                                  ColumnTransformer(transformers=[('pipeline-1',
                                                                   Pipeline(steps=[('simpleimputer',
                                                                                    SimpleImputer()),
                                                                                   ('minmaxscaler',
                                                                                    MinMaxScaler())]),
                                                                   <sklearn.compose._column_transformer.make_column_selector object at 0x0000027DBB9E1C48>),
                                                                  ('pipeline-2',
                                                                   Pipeline(steps=[('simpleimputer',
                                                                                    SimpleImputer(fill_value='missing',
                                                                                                  strategy='constant')),
                                                                                   ('onehotencoder',
                                                                                    OneHotEncoder(handle_unknown='ignore'))]),
                                                                   <sklearn.compose._column_transformer.make_column_selector object at 0x0000027DCA939D88>)])),
                                 ('selectfrommodel',
                                  SelectFromModel(estimator=LogisticRegression(solver='saga')))])),
                ('logisticregression', LogisticRegression(n_jobs=-1))]). Check the list of available parameters with `estimator.get_params().keys()`.

How do I access the max_iter parameter of LogisticRegression(), used as estimator in SelectFromModel() from GridSearchCV param_grid?

gridsearchcv scikit-learn python
1 Answers

Your selectfrommodel is nested under the lr_pipe. Following the error's suggestion to run lr_pipe.get_params().keys() should clue you in to that. It looks like you need pipeline__selectfrommodel__logisticregression__max_iter. (And maybe consider skipping make_pipeline and make_column_transformer, so you can give shorter names to the steps? You could also make one three-step pipeline instead of the nested pipes fs_pipe and lr_pipe.)

Ben Reiniger0
Related