[SOLVED] I am trying to compute the partial AUC by using the option max_fpr in roc_auc_score. However, the result does not convince me. Here's an example:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.metrics import roc_auc_score, roc_curve, auc
score = [1,2,3,4,4,5,5,5,6,6,7,7,8,9,10]
default = [0,0,1,0,0,0,1,0,0,0,1,1,0,0,1]
df = pd.DataFrame({"score":score, "default":default})
fpr = {}
tpr = {}
roc_auc = {}
partial_auc = {}
fpr, tpr, thresholds = roc_curve(df.default, df.score)
roc_auc = auc(fpr, tpr)
partial_auc = roc_auc_score(df.default, df.score, max_fpr=0.2).round(3)
The partial AUC I get here is 0.556 while the total one is 0.66. This looks weird especially if I plot it.
plt.figure()
lw = 2
plt.plot(fpr, tpr, color='purple',
lw=lw, label='auc='+str(roc_auc)+', partial_auc='+str(partial_auc))
plt.plot([0, 1], [0, 1], color='navy', lw=lw, linestyle='--')
plt.axvline(x=0.2, color='grey', linestyle='--')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receiver operating characteristic')
plt.legend(loc="lower right")
plt.show()
How come the partial AUC from 0 to 0.05 can be 0.556?
EDIT - I found the reason. The partial AUC I get is scaled with respect the minimum and maximum AUC I can get till the given max_frp, in order to have 0.5 if it is equal to the minimum (0.5max_frp**2) and 1 if it is equal to the maximum (1max_frp).
So partial_auc = 0.5 * (1 + (partial_auc - min_area) / (max_area - min_area))