yolo v1: why we weigh localization error more than classification error

Question

I'm reading yolo v1 paper. I am trying to understand this part of the paper:

"We use sum-squared error because it is easy to optimize, however it does not perfectly align with our goal of maximizing average precision. It weights localization error equally with classification error which may not be ideal."

In the loss function, $\lambda_{coord}$ parameter is used to weigh localization loss more. It seems like the aim is to maximize average precision.

By weighing localization error more, I feel like it implies that localization error plays more important role in average precision. But doesn't both classification and localization error both play equally important role? (as precision also depends on the predicted class). Any elaboration in this regard would be much appreciated.

Answer 1

Both components of the loss are important, but one cannot compare their numerical values a priori since they describe very different aspects, as in the proverbial apples and oranges. Consider the component related to the size of the bounding box: $$\lambda_\text{coord} \sum_{i = 1}^{S^2} \sum_{j = 1}^{B} \mathbb 1_{ij}^\text{obj} \left[\left(\sqrt{w_{ij}} - \sqrt{\hat w_{ij}}\right)^2 + \left(\sqrt{h_{ij}} - \sqrt{\hat h_{ij}}\right)^2\right].$$ Here the widths and heights are fractional with respect to the dimensions of the full image. Imagine that instead of the original dataset, we trained the neural network on 50% central crops by area. Then all these $w$ and $h$ would get scaled by $\sqrt 2$ and this component of the loss would increase by factor 2. This indicates that the numeric value of the localization loss is not meaningful by itself. Instead, $\lambda_\text{coord}$ is a hyperparameter that controls its relative importance. I suspect that the authors of the YOLO paper have tried different values for it and found that $\lambda_\text{coord} = 5$ gave a good mAP on the PASCAL VOC dataset.