3
$\begingroup$

I have a data set of photos containing an object in each of them. I want to find out the coordinates of rectangle enclosing the object.

Note that each photo contains exactly 1 object (for example, if there is a pair of shoes in the photo it is to be treated as one object), and the photos are taken in a simple white background. But the images do not contain one class of objects, the object can be anything.

I have a training set, consisting of photos, and the coordinates of the rectangle enclosing the object for these photos. And I want to find the coordinates of the enclosing rectangle, given a new photo (exactly 1 object, photos taken in simple white background).

I searched a lot for a method to do so, and found resources for achieving localization with classification, but neither do I want to classify the objects nor do I have class labels in my training set.

I also thought edge detection and object segmentation methods could be useful.

However, I feel that my task is much simpler since I know that I have to localize only 1 object in an image and the background is also simple, so there must be some simple methods I am overlooking.

Any guidance is much appreciated, and I am relatively new to machine learning so I would be grateful for guidance to implement the appropriate technique.

$\endgroup$
  • 1
    $\begingroup$ Off the top of my head it seems like anomaly detection or a simple clustering approach might be of interest here. Assuming that the single object you want to localize in the image presents one coherent shape (or close to it) that differs from the white background, so the "not purely white" observations (pixels/segments?) would form a cluster that needs to be detected. Note however that I'm fairly unfamiliar with image processing problems. $\endgroup$ – Rickyfox Feb 8 at 10:14
  • $\begingroup$ @Rickyfox Thanks a lot for replying. While I may use an unsupervised learning model like a clustering approach, it needs to be noted that the background is not 100% pixel-perfect white (they are photos clicked against a white background). Also I have training data, and I don't think discarding that and using clustering would give me good accuracy. Sorry to reject your edit, but I think completely unsupervised learning algorithms may not be the answer here. $\endgroup$ – Swapnil Rustagi Feb 8 at 10:34
  • $\begingroup$ Then I must have misunderstood you when you wrote "but neither do I want to classify the objects nor do I have a training set for classification." . $\endgroup$ – Rickyfox Feb 8 at 11:08
  • $\begingroup$ @Rickyfox I meant I have a training set. But it only contains the coordinates of the bounding box, not the classes of objects detected. $\endgroup$ – Swapnil Rustagi Feb 8 at 11:22
  • $\begingroup$ @Rickyfox I have edited my post to make it clearer that I do have a training set, but not class labels. $\endgroup$ – Swapnil Rustagi Feb 8 at 11:55
2
$\begingroup$

If photos are taken in a simple white background, and the object appearance are pretty distinguishable from the background. You do not really have to do as heavy as deep learning based method.

The task might fall into multiple aspects in computer vision, for example, foreground/background segmentation using Markov Random Field / Conditional Random Field / GraphCut.

If insisting using deep learning method, a look into the saliency detection topic might be helpful. This is a widely studied area with both traditional and deep learning methodology.

$\endgroup$
1
+50
$\begingroup$

There is nothing inherent about coupling localization with classification. For example - you could, in principle, train a neural network to predict for a given image four values, representing the bounding box of an "object" in that image. If you have enough data and you engineer the network properly, this could actually work.

More generally, people have sometimes referred to this problem of a class-generic object detector as "objectness". You can think of it as a foreground object segmentation task, where "object" is interpreted rather generally, so these approaches don't use specific class labels. Two concrete examples to get you started: the Objectness Measure quantifies how likely a given image window is to contain an object of any class, as opposed to backgrounds. To process an image, they have some approach for sampling a few candidate windows, and each window is evaluated based on their objectness measure. Another project that seems more up-to-date and less computationally heavy is Pixel Objectness. They produce a pixel-level mask for all "object-like" regions, which are not restricted to the object types encountered during training.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.