Why has the Auxiliary loss fallen out of favor? In deep learning, at one point, there were many deep models that utilize multiple softmax loss so that the gradient can flow better at the beginning of the neural net, such as Google Inception Net and PSPNet. However, recently, I haven't seen any network using this technique. Is there an alternate strategy now?

  • $\begingroup$ If the main purpose was to assist back-propagation, then you might be interested in residual networks. I can't say for sure, though, since I'm not familiar with auxiliary loss or its history. $\endgroup$ – Sycorax Feb 19 at 2:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.