OOB evaluation with metrics other than accuracy, e.g., F1 or AUC

user_12918630 0
Viewed 225

I am training random forest on the imbalanced dataset, accuracy isn't informative. I want to avoid cross validation and use out of bag (OOB) evaluation instead. Is it possible in sklearn (or in python in general) to evaluate out of bag (OOB) F1 or AUC instead of OOB accuracy?

I didn't find a way to do it on these pages:

https://scikit-learn.org/stable/auto_examples/ensemble/plot_ensemble_oob.html#sphx-glr-auto-examples-ensemble-plot-ensemble-oob-py https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html

Or should I just calculate f1 and AUC for the averaged predictions (or majority vote for classification) in oob_decision_function_?

auc cross-validation scikit-learn classification random-forest
1 Answers

From the source, you can see that the accuracy calculation is hard-coded, so you won't be able to just get another score by setting some parameter.

But, as you say, the oob predictions are available, so making the final computation yourself isn't difficult.

user_104958930
Related