How do I utilize the MAP eval metric in Catboost to calculate Mean Average Precision?

user_6875778 0
Viewed 535

I have been using a custom metric for Precision-Recall AUC in Catboost. However, it iterates slow and is incompatible with GPU. I see Catboost has a metric "MAP" for Mean Average Precision which is what I need for my (binary) classification model.

When I fit my Catboost Model using MAP as the eval metric I get the error:

CatBoostError: c:/program files (x86)/go agent/pipelines/buildmaster/catboost.git/catboost/private/libs/target/data_providers.cpp:269: Groupwise loss/metrics require nontrivial groups

After googling this error (specifically the "Groupwise loss/metrics require nontrivial groups" portion) I have had no luck resolving the issue.

Looking through the Catboost documentation it seems there may be some additional arguments I need to use in order for MAP to work properly as an eval metric. But I do not understand how to implement them.

How do I use MAP as my eval metric in Catboost for a binary classification problem?

catboost python
1 Answers

There is no metric MAP on Catboost. But to use average precision as metric you can use eval_metric="PRAUC:use_weights=false which has the same meaning with scikit-learn average_precision and MAP on XGboost. This metric is implemented since catboost 0.32.1 version.

user_93283100
Related