Change default RandomForestClassifier's "score" function when fitting the model?

user_708366 0
Viewed 2374

I perform the fitting operation using RandomForestClassifier from sklearn:

clf.fit(X_train,y_train,sample_weight=weight)

I don't know how to change the evaluation metric, which I assume it's simply accuracy here.

I'm asking this because I've seen that with the XGBOOST package you can precisely specify this metric. Example:

clf.fit(X_train, y_train, eval_metric="auc", eval_set=[(X_eval, y_eval)])

So, my question is: could I do the same with RandomForestClassifier from sklearn. I need to base my performance on AUC metric.

scikit-learn random-forest
3 Answers

I don't think you can change the metric used by the score method of RandomForestClassifier.

But this code should give you the auc:

from sklearn.metrics import roc_auc_score
roc_auc_score(y_eval, clf.predict_proba(X_eval))
user_24476560

As Guiem Bosch mentioned, the best way to get a different scoring method is by GridSearchCV. Indeed RandomForestClassifier has accuracy as scoring method.

However, I am not quite sure as to what you mean exactly with your question. You can always check the classifier's other scoring methods by running different scoring functions on the test set after the classifier is fitted by importing them from sklearn.metrics.

Be cautious here though:

  • With GridSearchCV and scoring=['roc_auc', 'recall'] etc. you will get the best classifier for the grid parameters, for each scoring metric you specify. For example, you will get the best classifier (set of hyperparameters) for 'roc_auc' score or for 'recall' score, based on which one you specify on the fitted parameter. But if you only have one set of hyperparameters and you are using GridSearchCV to just obtain different scoring methods, you can do this better with individual modules from sklearn.metrics.

  • If you mean though that you want your classifier to be optimised based on a different method, then you should check the criterion parameter.

user_93907230
Related