How can the mAP metric be meaningful for non-exhaustively labeled datasets (such as YouTube BoundingBox)?

Question

I am interested in reproducing the object detection results found in the whitepaper describing the YouTube BoundingBox dataset (https://arxiv.org/pdf/1702.00824.pdf). What I don't understand is how the authors can use the Mean Average Precision (mAP) metric to evaluate/train their network when using YouTube BoundingBox? As I understand it, precision is evaluated for object detection as:

precision = true_positives / (true_positives + false_positives)

While this makes a lot of sense for exhaustively labeled datasets like COCO, YouTube BoundingBox only tracks a single object over a few frames. To see an example of this, observe the zebra example in Figure 1 of the paper I link to above. While there are 5+ visible zebras in the frame, only one has a labeled bounding box. This means that even if the detection network was working exactly as a human would (observing and marking all of the zebras as zebras), it would have a low mAP as all of the non-labeled zebras would be considered as false positives.

Am I misunderstanding mAP, the training/evaluation method the paper used, or something else? If I am correct then it is pretty weird that Faster-RCNN can learn to recognize only the labeled object and ignore non-labeled objects. Perhaps this means that it learns the kind of objects that the human annotators were initially drawn to.

Answer 1

Basically, when calculating metrics, it pretends those unlabeled zebra no exist. (assuming COCO might also misses a 90% hidden zebra).

Eventually, the model might predict some unlabeled zebra as zebra, while the metrics think it's wrong. (which is not ideal) The hope is that, most frames are single object, hence labeled correctly. And the model, like human, learns to ignore some wrong knowledge.