Evaluation of multiple models using GridSearchCV and f2 score

user_14548086 0
Viewed 388

I am trying to classify some machine learning models in a binary classification using GridSearchCV. I want to classify the models with respect of score, best parameters and f2 score.

For the score and best parameters I used the code

scores = []

for model_name, mp in model_params.items():
    clf =  GridSearchCV(mp['model'], mp['params'], cv=5, return_train_score=False)
    clf.fit(X_test, y_test)
    scores.append({
        'model': model_name,
        'best_score': clf.best_score_,
        'best_params': clf.best_params_
    })
    
df = pd.DataFrame(scores,columns=['model','best_score','best_params'])

Which gives the best_score and best_params for all the models, but I am not able to do it find the f2 score of all the models. What should be the code?

scikit-learn machine-learning python
2 Answers
from sklearn.metrics import make_scorer,fbeta_score
def f2_func(y_true, y_pred):
    f2_score = fbeta_score(y_true, y_pred, beta=2)
    return f2_score

def my_f2_scorer():
    return make_scorer(f2_func)

clf = GridSearchCV(svm.SVC(), parameters, cv=10, scoring=my_f2_scorer())
user_168150870

You may want to have a look at clf.cv_results_ which holds the scores and other benchmark informations for all models searched (see also here)

user_60473570
Related