What is auxiliary loss that is mentioned in the PSPNet(Pyramid Scene Parsing Network) paper link ?

I'm quoting the part of the paper down below

An example of our deeply supervised ResNet101 [13] model is illustrated in Fig. 4. Apart from the main branch using softmax loss to train the final classifier, another classifier is applied after the fourth stage, i.e., the res4b22 residue block. Different from relay backpropagation [32] that blocks the backward auxiliary loss to several shallow layers, we let the two loss functions pass through all previous layers. The auxiliary loss helps optimize the learning process, while the master branch loss takes the most responsibility. We add weight to balance the auxiliary loss.

My question is how does this auxiliary loss work and how does it help in training process. What is its work in the network ?


I'm not totally sure about the use of the auxiliary loss in the PSPNet but in general such a auxiliary loss is used in networks with many layers.

Such a auxiliary loss helps to reduce the vanishing gradient problem for earlier layers, stabilizes the training and is used as regularization. It's only used for training and not for inference.

GoogLeNet also used auxiliary losses: https://arxiv.org/abs/1409.4842

  • $\begingroup$ I understood that. But after some thought I have understood it intuitively. The auxiliary loss will add extra gradient flow during backpropogation, thereby helping to reduce gradient vanishing problem, training stability as you mentioned. I wanted even more clearer explanation if there is any. That is why I posted here. Thank you for your answer. $\endgroup$ – papabiceps Jul 8 '18 at 19:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.