How does the YOLO network create boundaries for object detection?

Question

I'm not sure I understood how the YOLO network works. If you look at the description, https://pjreddie.com/darknet/yolo/, it appears to me that all is done thanks to convolution only. You end up with NxNxM results, where each M array contains a couple bounding boxes, classes, etc. Apparently, from the way I understand it, is each array in those M wide cells you will have 4 values that tell the center position (but only if it lies in that grid cell), then the width and the height of the bounding box. This way it seems that the bounding box is "around" the cell.

See the explanation at https://medium.com/diaryofawannapreneur/yolo-you-only-look-once-for-object-detection-explained-6f80ea7aaa1e

But each cell is an aggregation of feature cells beneath that very same cell, so how is it possible that it may encode the size of the bounding box outside of it?

Answer 1

No, YOLO doesn't use only convolutions. As you can see in the architecture diagram, there are two fully-connected layers between the main convolutional part and the final convolutional output.

These fully-connected layers allow it to essentially do regression on the bounding box center coordinates as well as the size and width which can range over the whole image. Please let me know if I haven't understood your question correctly!

Answer 2

Disclaimer: This answer contains details about the SSD object detector net rather than YOLO, but the core idea is similar.

You are right that each cell in a feature map produced by a convolutional network represents some information about the corresponding region of the image. On the illustration bellow you can see a 4x4x256 feature maps visualized and the corresponding 4x4 grid of the original image:

However, that is not the same as the receptive filed of each of those cells: Receptive field would be the region of the image, whose pixels were used to compute the value of the cell. The exact receptive field depends on the exact size and number of your convolutional layers, but it is usually much larger than the image cell. That means the cell contains also information about its surroundings and it can predict also bounding boxes larger than itself.

Now regarding the mechanism of individual bounding box predictions (which are also realized as convolutions): You predefine some number of anchor boxes (bottom right in the image) defined as width and height, centered in the center of the cell. For each of them, the network predicts 4+num_classes numbers:

• vertical offset of the center
• horizontal offset of the center
• width difference
• height difference
• probability of each class that an object of this class is present in this bounding box (these are normalized via softmax)

Therefore, $M = \mathrm{num\_anchors}\, \cdot\, (4+\mathrm{num\_classes}) $.