Say I have a neural network that classifies images by training to minimise cross-entropy loss with one-hot encoded training labels. It is often seen that such neural networks are 'overconfident', with the softmax of the final layer giving a lot of mass to one class even if it is wrong. However the ultimate concern is usually accuracy, so why is this seen as such a problem? If I understnad correctly, the softmax of the final layer is just some numbers and has no meaningful probabilistic interpretation anyway?
There must be more to the story here, since techniques like label smoothing that encourage better calibration also often lead to better performance in test accuracy, but I don't really understand why that should happen?
However the ultimate concern is usually accuracy
Accuracy has major limitations for both balanced and imbalances problems. Please see this and this and the Frank Harrell blog posts described within: (1) (2). $\endgroup$