In the context of cross-entropy loss objectives for neural networks, I tend to think of label smoothing from the standpoint of directly manipulating the labels. This blog post explains how doing so is equivalent to adding an extra loss term, that is the the cross-entropy loss between the uniform noise distribution and the model output distribution:
I can follow this just fine.
But when I look at Regularizing Neural Networks By Penalizing Confident Output Distributions it talks about the extra term being a KL divergence instead (see section 3.2). In the latter case, I can't make the connection.
I have a related question on this.
But now I'm just totally confused about whether label smoothing is equivalent to adding a cross entropy term, or if it's equivalent to adding a KL divergence term.