I've been reading about how object detection models are evaluated. It seems that the metric most often used is AP. But I have stumbled upon 2 different approaches that I think mean completely different things:
In the first approach, we use different IOU thresholds for defining what constitutes a TP and FP (wrt to bounding boxes) and compute a pair of points (precision, recall) for every threshold (by going through every detection). We compute the area under the curve and we have the Average Precision of the model, for a given class.
The second approach is stranger: we have a fixed threshold say 0.5 and go through every detection for a class, if the IOU of that detection > 0.5 then is this TP, otherwise is a FP. For every detection, we compute the precision based only on the predictions made up until a detection and the recall using information of the total number of elements of particular class in the dataset. In this scenario, we have as many (precision, recall) points as we have detections, and as we move forward in the detections the recall gets closer to 1, and the precision zizags. In this version of AP we compute the area under the curve of this graph. They call this AP_0.5, you can also compute others by changing the threshold.
I've trouble reconciling the 2 methods, the first one makes sense to me as a way of evaluating a model, but the second one, not so much. What does AP_0.5 tell us, why not just computing precision and recall for the threshold of 0.5 (using all detections) and compute an F1 score for each class?
Thank you for the help in advance!!