I am interested in reproducing the object detection results found in the whitepaper describing the YouTube BoundingBox dataset (https://arxiv.org/pdf/1702.00824.pdf). What I don't understand is how the authors can use the Mean Average Precision (mAP) metric to evaluate/train their network when using YouTube BoundingBox? As I understand it, precision is evaluated for object detection as:
precision = true_positives / (true_positives + false_positives)
While this makes a lot of sense for exhaustively labeled datasets like COCO, YouTube BoundingBox only tracks a single object over a few frames. To see an example of this, observe the zebra example in Figure 1 of the paper I link to above. While there are 5+ visible zebras in the frame, only one has a labeled bounding box. This means that even if the detection network was working exactly as a human would (observing and marking all of the zebras as zebras), it would have a low mAP as all of the non-labeled zebras would be considered as false positives.
Am I misunderstanding mAP, the training/evaluation method the paper used, or something else? If I am correct then it is pretty weird that Faster-RCNN can learn to recognize only the labeled object and ignore non-labeled objects. Perhaps this means that it learns the kind of objects that the human annotators were initially drawn to.