Why the best parameter found by grid search performs worse than default even though the default was included in the search space

Viewed 90

I have used the steps given in Analytics Vidhya. In one of the steps it has the following:

param_test4 = {'subsample':[i/10.0 for i in range(6,10)],
               'colsample_bytree':[i/10.0 for i in range(6,10)]
                }
gsearch4 = GridSearchCV(estimator = XGBClassifier( learning_rate =0.1, 
                                                   n_estimators=177, 
                                                   max_depth=4,
                                                   min_child_weight=6, 
                                                   gamma=0, 
                                                   subsample=0.8, 
                                                   colsample_bytree=0.8,
                                                   objective= 'binary:logistic', 
                                                   nthread=4, 
                                                   scale_pos_weight=1,seed=27), 
                        param_grid = param_test4, 
                        scoring='roc_auc',
                        n_jobs=4,
                        iid=False, 
                        cv=5)
 gsearch4.fit(train[predictors],train[target])
 gsearch4.grid_scores_, gsearch4.best_params_, gsearch4.best_score_

I have tried this code (on my own data). and before doing grid search for parameters in param_test4, I did fit the model for subsample=0.8, and colsample_bytree=0.8. Even tho these two values are amongst the list of parameters set in the param_test4, the output of grid search performs worse than just using subsample=0.8, and colsample_bytree=0.8 directly. I was wondering if anyone has seen this problem/situation before.

Thanks

0 Answers
Related