CNN high confidence on wrong classifications I've trained a CNN for alphanumeric digits classifcation on about 2M image samples.
The network is rather simple:
conv->relu->maxpool->conv->relu->maxpool->flatten->dense->relu->dense->softmax

Accuracy is very good, I get over 99% on the validation set, but what troubles me is the fact that the network's prediction confidence is always very high (98%+) even for samples that are misclassified.
I thought it could be a problem of overfitting, so I saved a checkpoint of the network every 2000 steps during training, but almost all of them show the same behaviour (saved for the first two checkpoints or so, where te network's accuracy is still not good enough).
I tried researching the problem, but I'm not even sure what's the most appropriate search query, so I didn't come up with much.
What other reasons might be for the network excessive confidence in its predictions?
 A: This is a fairly common occurrence with neural network models. My answer here is a conjecture, but I suspect that the reason this occurs is related to the problem of adversarial examples. Neural networks tend to have very steep transitions between classes, so examples can be near the boundary between two classes, and yet be dramatically misclassified. Adversarial examples take advantage of that by finding small modifications to an image from class $a$ that cause it to be classified as class $b$.
From this perspective, the problem appears to be one of regularization, but I don't think that anyone has a great sense of the specific sequence of steps to take so that ambiguous or unclear inputs are not strongly classified in any category.
A: Excessive confidence implies one of two things: firstly it could mean your network is really good at isolating the correct prediction, but this is unlikely due to the still high confidence on incorrect classifications and the fact that the network isn't particularly deep; secondly it could mean that the weights to your last dense layer are incredibly high, which forces the predictions post softmax to effectively 'mute' all other classes but the one with highest confidence.
Introducing some regularisation in your dense layers such as L2 regularisation results in the penalizing large weights. This has the effect of preventing overfitting and overconfidence. This will increase your loss (because confidence will decrease slightly) but other metrics that use the hard predictions (such as accuracy) will not decrease.
The network may also benefit from some dropout, but start out with some light L2 regularisation in your dense layers (start out with a small L2 weight of 0.05 and then adjust from there based on cross validation).
Also on a side note, a validation accuracy of 99% is really high. Are you performing any data augmentation? And is that augmentation happening before or after the validation split, because if it's occurring before you will have fallen into the trap of data leakage. Make sure you always do the validation split before any preprocessing (except for removal of samples).
A: The modern deep Neural networks tend to be overconfident.
you can also consider computing uncertainty estimates along with the confidence.
https://arxiv.org/abs/1706.04599
https://arxiv.org/abs/1612.01474
