I recently came across this paper in section 3.2 it talks about label smoothing loss and how it's equivalent to s equivalent to adding the KL divergence between the uniform distribution u and the network’s predicted distribution p.

$L = -\sum{log(p)-D_{KL}(u, p)}$

$u$ is the uniform distribution. On the other hand we know that the label smoothing has a distribution of $1-\epsilon$ for the target and $\epsilon/(V-1)$ for the rest of the labels. Given this I can't infer the formula above from the label smoothing value. Can anyone knows how to solve this problem?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.