1
$\begingroup$

I'm looking to build a computer vision model that can detect the presence or absence of several classes of object within an image, but the localization of these objects is not required. The end product would look something like the feature in Apple's Photos app search: I enter a term like "dog," and images containing dogs are returned, but the location of dogs within each image is not conveyed. In short, I need to know whether an image contains an image of class $X$, but I don't need to know where $X$ is.

I have looked into object detection models such as YOLO and Faster R-CNN, which both detect and localize objects. In fact, I have had a good deal of success simply by fine-tuning an off-the-shelf, pre-trained YOLOv5 model. But because so much work goes into fitting the bounding boxes predicted by object detection models, I have begun to wonder: could I see performance gains in the "object-presence task" if I adopted a model that wasn't concerned with the localization aspect of object detection? What would the name be for such a task? Might a different model architecture be better suited for this task?

The closest references I have found on this subject are from this DataScience Stack Exchange question, which mentions training a detection model without annotated bounding boxes. The author of this question, unlike me, is still interested in object localization. The linked paper Is Object Localization For Free? appears promising, but unlike the paper, my training dataset does have object-level bounding box annotations.

I would be happy to read any literature on the subject.

$\endgroup$

1 Answer 1

2
$\begingroup$

You could just train a classification model, since this sounds like a classification problem rather than a detection problem. Of course, it wouldn't surprise me if a pretrained object detection or semantic segmentation model does better, just because multi-task learning tends to improve performance. I would suggest starting off with the pretrained backbone of some detection or segmentation network, then add on a classification head which you train on your data, before possibly fine-tuning the entire net.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.