ScikitLearn GridSearchCV and pipeline using different methods

user_13552641 0
Viewed 394

I am trying to evaluate these Machine Learning methods to the same data using GridSearchCV and pipeline, when I vary the parameters in the same method it works, but when I put multiple Methods it gives an error

pipe_steps = [
    ('scaler', StandardScaler()), 
    ('logistic', LogisticRegression()),
    ('SVM',SVC()),
    ('KNN',KNeighborsClassifier())]
check_params={
    'logistic__C':[1,1e5],
    'SVM__C':[1,1e5],
    'KNN__n_neighbors':[3,5],
    'KNN__metric':['euclidean','manhattan']
    }
pipeline = Pipeline(pipe_steps)
GridS = GridSearchCV(pipeline, param_grid=check_params)
GridS.fit(X, y)
print('Score %3.2f' %GridS.score(X, y))
print('Best Fit')
print(GridS.best_params_)

gives the error message on pipeline line below

TypeError                                 Traceback (most recent call last)
<ipython-input-139-75960299bc1c> in <module>
     13     }
     14 
---> 15 pipeline = Pipeline(pipe_steps)
     16 
     17 BCX_Grid = GridSearchCV(pipeline, param_grid=check_params)

C:\ProgramData\Anaconda3\lib\site-packages\sklearn\pipeline.py in __init__(self, steps, memory, verbose)
    133     def __init__(self, steps, memory=None, verbose=False):
    134         self.steps = steps
--> 135         self._validate_steps()
    136         self.memory = memory
    137         self.verbose = verbose

C:\ProgramData\Anaconda3\lib\site-packages\sklearn\pipeline.py in _validate_steps(self)
    183                                 "transformers and implement fit and transform "
    184                                 "or be the string 'passthrough' "
--> 185                                 "'%s' (type %s) doesn't" % (t, type(t)))
    186 
    187         # We allow last estimator to be None as an identity transformation

TypeError: All intermediate steps should be transformers and implement fit and transform or be the string 'passthrough' 'LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,
                   intercept_scaling=1, l1_ratio=None, max_iter=100,
                   multi_class='warn', n_jobs=None, penalty='l2',
                   random_state=None, solver='warn', tol=0.0001, verbose=0,
                   warm_start=False)' (type <class 'sklearn.linear_model.logistic.LogisticRegression'>) doesn't

Thanks

gridsearchcv scikit-learn pipeline python
2 Answers

You need to split the pipeline into multiple pipelines, for that I have a solution that requires a list of grid params that determines each step of the pipeline.

pipeline = Pipeline([
    ('transformer', StandardScaler(),),
    ('model', 'passthrough',),
])

params = [
    {
        'model': (LogisticRegression(),),
        'model__C': (1, 1e5,),
    },
    {
        'model': (SVC(),),
        'model__C': (1, 1e5,),
    },
    {
        'model': (KNeighborsClassifier(),),
        'model__n_neighbors': (3, 5,),
        'model__metric': ('euclidean', 'manhattan',),
    }
]

grid_Search = GridSearchCV(pipeline, params)

With this strategy you can define the steps of the pipeline dynamically.

user_73959110

Your problem is not the hyperparameters as they are defined correctly. The problem is that all intermediate step should be transformers, as the error indicates. in your pipeline SVM is not a transformer.

see this post

user_76982970
Related