I have used the steps given in Analytics Vidhya. In one of the steps it has the following:
param_test4 = {'subsample':[i/10.0 for i in range(6,10)],
'colsample_bytree':[i/10.0 for i in range(6,10)]
}
gsearch4 = GridSearchCV(estimator = XGBClassifier( learning_rate =0.1,
n_estimators=177,
max_depth=4,
min_child_weight=6,
gamma=0,
subsample=0.8,
colsample_bytree=0.8,
objective= 'binary:logistic',
nthread=4,
scale_pos_weight=1,seed=27),
param_grid = param_test4,
scoring='roc_auc',
n_jobs=4,
iid=False,
cv=5)
gsearch4.fit(train[predictors],train[target])
gsearch4.grid_scores_, gsearch4.best_params_, gsearch4.best_score_
I have tried this code (on my own data). and before doing grid search for parameters in
param_test4, I did fit the model for subsample=0.8, and colsample_bytree=0.8. Even tho these two values are amongst the list of parameters set in the param_test4, the output of grid search performs worse than just using subsample=0.8, and colsample_bytree=0.8 directly. I was wondering if anyone has seen this problem/situation before.
Thanks