FROC Implementation in Python

user_5594682 0
Viewed 767

I have one system which predicts the location of boxes in the page. In the image below, the green rectangular is the correct location (label) and the red ones are the predicted. It is clear that the best performance of the system is when all red rectangular were located almost on the same location of green rectangular. My question is that how I can plot FROC (free-response ROC) curve for such a system to obtain the sensitivity of the predictor. How in this kind of problem True Positive, False Positive and etc are defined.

enter image description here

roc free python
2 Answers

It's kind of late, but maybe I can help you. First of all, you would need to define an IoU threshold T. Predicted bounding boxes (bbox) that overlap ground truth with an IoU > T and the class (assuming you have multiple objects you want to detect) associated with each bbox is the same, than you count this prediction as a TP. The rest of the predictions that do not overlap ground truth with a sufficient IoU value, or the class is different, are considered FP.

To summarize:

  • IoU(Pred, GT) > T AND (Pred_class == GT_class) => TP
  • IoU(Pred, GT) < T OR (IoU(Pred, GT) > T AND (Pred_class != GT_class)) => FP

To compute Sensitivity you don't need to count the FN, since you already now the total number of positives, i.e ground truth bbox for a particular class. Having TP and FP, you could also compute Precision.

Regarding FROC, you may find this repo useful.

user_68934220

You may want to check out this repo. You can obtain a figure like this one: enter image description here

Disclaimer: the code was tailored for a paper, and might need to be adapted for your application

user_94926730
Related