I am quite confused on the difference between Keras' AUC(curve='PR') and Scikit-learn's average_precision_score. My objective is to compute the Area Under the Precision-Recall Curve (AUPRC), for both Scikit-learn and Keras models. However, these two metrics yield vastly different results!
Did I miss something out on the TensorFlow-Keras documentation at https://www.tensorflow.org/api_docs/python/tf/keras/metrics/AUC, with regards to the use of the AUC function?
As stated in the Scikit-learn documentation, they use a different implementation method:
References [Manning2008] and [Everingham2010] present alternative variants of AP that interpolate the precision-recall curve. Currently, average_precision_score does not implement any interpolated variant. References [Davis2006] and [Flach2015] describe why a linear interpolation of points on the precision-recall curve provides an overly-optimistic measure of classifier performance. This linear interpolation is used when computing area under the curve with the trapezoidal rule in auc.
In the average_precision_score function documentation, you can also read:
This implementation is not interpolated and is different from computing the area under the precision-recall curve with the trapezoidal rule, which uses linear interpolation and can be too optimistic.
I encourage you to look in detail at the different functions and their descriptions available in the metrics module. I also highly recommend to read the related paper.
Lastly, there's also a potentially interested thread here: [AUC] result of tf.metrics.auc doesnot match with sklearn's.
