Pipeline for more than 2 classifiers

user_17750082 0
Viewed 17

I am trying to build an ensemble using Knn and random forest classifiers.

steps = [('scaler', StandardScaler()),
     ('regressor', VotingClassifier(estimators=[
     ('knn', KNeighborsClassifier()), 
     ('clf', RandomForestClassifier())]))]
pipeline = Pipeline(steps)
parameters = [{'knn__n_neighbors': np.arange(1, 50)}, {
             'clf__n_estimators': [10, 20, 30],
             'clf__criterion': ['gini', 'entropy'],
             'clf__max_features': [5, 10, 15],
             'clf__max_depth': ['auto', 'log2', 'sqrt', None]}]
X_train, X_test, y_train, y_test = train_test_split(X, y.values.ravel(),
test_size=0.3, random_state=65)
cv = GridSearchCV(pipeline, param_grid=parameters)
cv.fit(X_train, y_train)
y_pred = cv.predict(X_test)

I have encourntered the following error while running the above code:

Invalid parameter knn for estimator Pipeline(steps=[('scaler', StandardScaler()), ('regressor',VotingClassifier(estimators=[('knn', KNeighborsClassifier()),('clf', RandomForestClassifier())]))]). Check the list of available parameters with estimator.get_params().keys()

Since I am new to machine learning I having difficulty in understanding the error.

knn pipeline random-forest python-3.x
1 Answers

I agree that the error message is not clear, but the error is raised because of the knn estimator in your VotingClassifier. Please check the VotingClassifier documentation:

voting : str, {'hard', 'soft'}, default='hard' If 'hard', uses predicted class labels for majority rule voting. Else if 'soft', predicts the class label based on the argmax of the sums of the predicted probabilities, which is recommended for an ensemble of well-calibrated classifiers. (...)

attributes : dict, default=None Dictionary mapping each estimator to a list of attributes to be extracted as predictors.

If None, all public estimator attributes are used.

Concretely, KNeighborsClassifier has no predict_proba method, thus it cannot work with the soft voting classifier. If you want to keep both estimators in your VotingClassifier, you should set voting='hard': VotingClassifier(estimators=[('knn', KNeighborsClassifier()), ('clf', RandomForestClassifier())],voting='hard')

Let me know if it helped.

user_167730630
Related