Can U-Net be used for counting objects?

Question

If I understand the U-Net paper correctly, the NN output is segmentation of known objects on the image from the background. In other words, the network will try to mark all the pixels which are part of the detected objects, but it won't distinguish between object instances. Is my understanding correct?

If yes, is there an approach which would allow me to differentiate between object instances in the image?

For example, if we have a photo of a street I would like to know not only which pixels are taken by cars, but also "enumerate" the said instances (even partly overlapping ones).

The reason I am asking is that I believe NN already learns what the object is and should therefore intrinsically "detect" the instances, however the U-Net input and output do not match this expectation directly (or I have misunderstood something - quite possible as I am still a newbie at ML).

Answer 1

U-net outputs a segmentation, i.e. a binary map. If you want to get the number of objects detected, the most straightforward approach would be to compute the connected components of your binary maps. But as you mention, overlapping object might be a problem. There are CNN specifically design for this task. For instance: http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Stewart_End-To-End_People_Detection_CVPR_2016_paper.pdf which uses LSTM (recurrent network) or https://arxiv.org/pdf/1705.07999.pdf which is designed for 3D. I hope this helps!