Regarding the first comment of the question:
- It Extract features from DarkNet-53 and then again use 53 Convolution networks throughout.
The network used to extract features is just DarkNet-53 as stated by the author in the YOLOv3 paper:
We use a new network for performing feature extraction [...]
It has 53 convolutional
layers so we call it.... wait for it..... Darknet-53!
Regarding the second comment of the question:
- At 3 different levels (at layers number: 82,96,106), there are feature maps created of size: 13, 26, 52 respectively with same depth 256. The features at 82 are upscaled till layer 96 and features from 96 upscaled till 106. (What are these? Feature Map size?)
Yes, the sizes of 13, 26 and 52, refer to the feature map dimensions at the prediction layers. These sizes are directly related to the grid cells as we are going to see in the following lines. Just a sidenote, the correct depth (number of feature maps) is 255 and not 256 (this will also be explained hereafter).
Grid cells
Grid cells were first explained in the original paper You Only Look Once. They are part of a regular grid (of simensions $S\times S$) into wich the input image is divided. Concretely, in YOLOv1 the author uses a grid of 7x7 cells. This is an example:
Images extracted from YOLO CVPR slides
The purpose of this type of grid (which is mantained in YOLOv3) is to serve as the output of the architecture $\Rightarrow$ $B$ bounding boxes are being predicted at each of the grid cells. Specifically, in YOLOv3, $B=3 \Rightarrow$ We are predicting 3 bounding boxes at each grid cell.
But how can this be encoded in the network?
The authors used what you mentioned in the question i.e. 3 different prediction layers/ grids composed of $13\times 13, 26\times 26$ and $52\times 52$ grid cells.
Now, we should remember that $B=3$ bounding boxes are being predicted at each of these grid cells. These boxes are encoded as a vector of $85$ features/ dimensions $\Rightarrow$ The depth (number of feature maps) needed is $85\times B = 255$, as you said in the question.
To visualize this, first we can focus on how a certain box of a certain cell is encoded:
- $\mathbf{C}$ : Is the objecteness of the associated bounding box (sort of a confidence measure of how likely a box is containing, or not, an object).
- $\mathbf{(t_x, t_y)}$ : Features related to the offsets of the bounding box center w.r.t. a corner of the grid cell.
- $\mathbf{(t_w, t_h)}$ : Features related to the dimensions of the bounding box.
- $\mathbf{p_1,...,p_{80}}$ : Conditional probabilities, $P(\text{Class}| \text{Object})$ , for each of the $80$ classes being considered in the paper.
By doing this we end up predicting a tensor of dimensions $S\times S\times B \times 85$, which represents the output of the architecture, as commented above:
However, we are predicting 3 bounding boxes for each grid cell $\Rightarrow$ the total output is the result of concatenating three of these tensors together, making a dimensions total of $S\times S \times 85 \times 3 = S\times S \times 255$.
Moreover, as you said in the question, YOLOv3 uses three of these regular grids as outputs $\Rightarrow$ We are predicting 3 of these $S_i\times S_i \times 255$ tensors at 3 different depths of the architecture.
Predictions and training with YOLOv3
Due to the 3 different grid resolutions $(S_i = \{13, 26, 52\})$, we will be predicting outputs at 3 different depths of the architecture. This is shown in the next image extracted from the great post "What’s new in YOLO v3?":
At these 3 different layers (outputs), the same object can be detected, however this is not very likely to happen. This is because, as said by the authors, each of these regular grids uses anchor boxes with different sizes $\Rightarrow$ The lowest resolution grid ($S=13$) uses bigger anchor boxes, and the highest resolution grid ($S=52$) uses smaller anchor boxes.
The purpose of this, is to be able of detecting smaller objects which are difficult to detect in a low resolution grid. This way, in order to detect bigger objects at he highest resolution grid, the difference w.r.t. these anchor boxes would be significant, difficulting the learning procedure of the network.
However, having a look at the label generation for VOC dataset:
Now we need to generate the label files that Darknet uses. Darknet wants a .txt file for each image with a line for each ground truth object in the image that looks like:
<object-class> <x> <y> <width> <height>
Where x, y, width, and height are relative to the image's width and height
There isn't distinction of scale for the ground-truth objects. This means that the network will be learning to predict the ground-truth bounding boxes at the 3 different grids $\Rightarrow$ A ground-truth object will have influence at the loss function of all 3 output grids.
This influence in learning is by backpropagation of the gradients of the 3 loss function w.r.t. the weights/ biases. One loss function for each output grid. However, note that, for example, not all the weigths/ biases will have an influence in the first and second grid loss functions.
This is because, for example, a change in the weights and biases values of one of the final layers won't affect the predictions of the first output grid $\Rightarrow$ The associated gradient will be zero.
