# Huber Loss on top of Cross Entropy

I know that the Huber loss is usually applied on top of the L2 loss in order to prevent exploding gradients. Does it make sense to use the Huber loss on top of the cross entropy loss, though? I have a feeling that it is not very sensible, but a more scientific argument will be much more persuasive.

However, if the noise is not normally distributed, you might get values which are very far from your prediction, causing a large error / gradient. So the Huber loss makes sense when you expect these extreme events to be more frequent. The L1 loss, which is close to the Huber loss, gives the maximum likelihood for the laplace distribution, which has tails of $O(e^{-|x|})$ as opposed to the normal distribution's $O(e^{-x^2})$.