I'm quite confused as to how I can calculate the AP or mAP values as there seem to be quite a few different methods. I specifically want to get the AP/mAP values for object detection.
All I know for sure is:
Recall = TP/(TP + FN), Precision = TP/(TP + FP)
For example, if I only have 1 class to evaluate, and say 500 test images. Each test image may have different number of predictions (bounding box proposals) but each image only has one ground-truth bounding box.
Image 1: [class, probability, x1, y1, x2, y2], [class, probability, x3, y3, x4, y4], [class, probability, x5, y5, x6, y6], [class, probability, x7, y7, x8, y8], ...
Image 2: [class, probability, x1, y1, x2, y2], [class, probability, x3, y3, x4, y4], ...
. . . (and so on)
*just an example, I made this up
I know that to get TP, we'd have to find the IOUs of each prediction and count the ones above a selected threshold such as 0.5 (if we have multiple predictions with IOUs above the threshold, do we only count once and treat the others as FP?).
This is where it puzzles me:
Would the TP+FP = # of predictions made for each image?
Since all test images have no negatives, TP+FN = 500?
Is it calculated per image, or per class?
Could someone let me know a step by step guide to get the AP/mAP based on my example? I find the most ambiguous part is whether we do it per image, or per class (i.e. 500 images all at once).
Most guides/papers I found are very targeted towards information retrieval. Would appreciate some help in this.
*Note: I am testing it on some custom dataset. I know PASCAL VOC has some code to do it, but I want to write the code myself, customised to my own data.