1
$\begingroup$

I'm implementing Yolo v3 in Pytorch,

Imagine we have an image with a large object in it, whem building targets, I'll have to assign the anchor box with highest IoU with this object for each scale, (So each scale will have and anchor box assigned to this object, my interpretation), the IoU for the lowest resolution grid 13x13 may be good, but let's say the IoU for the 52x52 resolution is too low, so, the offsets the network will need to learn for that anchor box fit the target for this scale are huge, the oposite case (small object) is also true, so, does it do not harm the performance for detecting the objects? I mean, if I plot the mean IoU over all scales it will not be high, because the 13x13 one will not detect small objects and the 52x52 one will not detect large objects. is this the case? what is the metric I'll use for see training performance if this mean does not looks a good option?

Would make sense to assign only one anchor box in the scale that it has the best IoU and do not assign an anchor box for the others scales? I mean drop the box for the rest and let only the 13x13 scale predict for large objects, 26x26 for medium and 52x52 for small?

Here is my implementation of the build targets function: (I followed Aladdin Persson's implementation)

In resume my questions are:

  • Is this correct?
  • What metric can I use to see if the training is going well?
  • Why this will not harm the performance?

Example of what I have mentioned above

In green the ground truth and in red the anchor box assigned to the object.

# boxes for an image -> [[Class,xCenter,yCenter,width,height],[Class,xCenter,yCenter,width,height]]

# S = [13,26,52]

# C = 20 # num of classes

# B = 3 # num of anchors per grid cell.


def create_targets(self,boxes,S,C):


  target_matrix = [torch.zeros((torch.div(self.total_num_of_anchors, 3, rounding_mode='floor'), S, S, 1+4+1)) for S in self.S] 

  #target_matrix
  #torch.Size([3, 13, 13, 6]) 
  #torch.Size([3, 26, 26, 6])
  #torch.Size([3, 52, 52, 6])

  for box in boxes:

    class_ = int(box[0])
    xCenter = float(box[1])
    yCenter = float(box[2])
    width = float(box[3])
    height = float(box[4])

    # IoU 
    iou_anchors = iou_width_height(torch.tensor((width,height)), self.anchor_boxes)
    # Highest IoU
    anchor_indices = iou_anchors.argsort(descending=True, dim=0)

    has_anchor = [False,False,False] 

    for anchor_index in anchor_indices:
      # 0/3 = 0 , 1/3 = 0, 2/3 = 0 i.e index 0,1,2 scale 1.
      # 3/3 = 1 , 4/3 = 1, 5/3 = 1 i.e index 3,4,5 scale 2.
      # 6/3 = 2 , 7/3 = 2, 8/3 = 2 i.e index 6,7,8 scale 3.

      # find the scale of the anchor.
      scale_index = torch.div(anchor_index, self.num_anchors_per_scale, rounding_mode='floor')  # 0, 1 or 2. ; anchor_index // self.num_anchors_per_scale 


      # 0%3 = 0 -> 1º anchor
      # 1%3 = 1 -> 2º anchor
      # 2%3 = 2 -> 3º anchor 
      # etc.
      # Find the index of the anchor, 1º 2º or 3º anchor box in that scale.
      anchor_on_scale = anchor_index % self.num_anchors_per_scale 

      S_scale = self.S[scale_index] 
         
      # grid cell where the object is.
      i, j = int(S_scale*yCenter), int(S_scale*xCenter) 

      # there is an anchor is this position?
      anchor_taken = target_matrix[scale_index][anchor_on_scale, i, j, 0] 

      if not anchor_taken and not has_anchor[scale_index]:
        # confidence score set to 1.
        target_matrix[scale_index][anchor_on_scale, i, j, 0] = 1 

        # converting coordinates for cell relative.
        x_cell, y_cell = S_scale*xCenter - j, S_scale*yCenter - i 
        width_cell, height_cell = width*S_scale, height*S_scale 

        # store coordinates and class
        target_matrix[scale_index][anchor_on_scale, i, j, 1:5] = torch.tensor([x_cell, y_cell, width_cell, height_cell]) 
        target_matrix[scale_index][anchor_on_scale, i, j, 5] = class_ 

        has_anchor[scale_index] = True

      # to be honest I did not understood very well this part
      # what is the difference if it's -1 or 0? , it will not be punished in the loss function if there is not an object assigned to this position.
      elif not anchor_taken and iou_anchors[anchor_index] > self.ignore_iou_thresh: 
        target_matrix[scale_index][anchor_on_scale, i, j, 0] = -1

  return tuple(target_matrix)

"""
$\endgroup$

0