1
$\begingroup$

Say I have a neural network that classifies images by training to minimise cross-entropy loss with one-hot encoded training labels. It is often seen that such neural networks are 'overconfident', with the softmax of the final layer giving a lot of mass to one class even if it is wrong. However the ultimate concern is usually accuracy, so why is this seen as such a problem? If I understnad correctly, the softmax of the final layer is just some numbers and has no meaningful probabilistic interpretation anyway?

There must be more to the story here, since techniques like label smoothing that encourage better calibration also often lead to better performance in test accuracy, but I don't really understand why that should happen?

Improve this question
$\endgroup$
7
  • $\begingroup$ The output of softmax is a probability vector, so it most certainly does have a probabilistic interpretation! $\endgroup$
    – Sycorax ♦
    Aug 14 at 13:02
  • $\begingroup$ @Sycorax Right but it doesn't really make a meaningful statement about probabilities given the data $\endgroup$
    – Danny Duberstein
    Aug 14 at 13:17
  • 1
    $\begingroup$ However the ultimate concern is usually accuracy Accuracy has major limitations for both balanced and imbalances problems. Please see this and this and the Frank Harrell blog posts described within: (1) (2). $\endgroup$
    – Dave
    Aug 14 at 13:20
  • $\begingroup$ @Dave Thanks for the reference. I guess what I said was imprecise, my point was more that a classifier in use would typically just output one classification, i.e. the argmax of the outputted probabiltiy vector. $\endgroup$
    – Danny Duberstein
    Aug 14 at 13:40
  • 1
    $\begingroup$ @DannyDuberstein That’s exactly what those links argue should not be done. Predicted probabilities matter. $\endgroup$
    – Dave
    Aug 14 at 13:43

1 Answer 1

Reset to default
3
$\begingroup$

Use high stakes to see this.

Say you are predicting the probability of death vs survival. Your model predicts that, if you proceed with a particular course of action, say skydiving, the probability of survival is $0.9999$, so a one-in-ten-thousand chance of dying. That makes the skydiving sound rather safe and worth the small risk for the extreme thrill. However, if the chance of dying is actually one-in-three, the model has mislead the skydiver into thinking this risky activity is low-risk. The skydiver might be willing to jump if the risk of death is one-in-ten-thousand but not one-in-three. That is, the model being overconfident about the probability of survival has led the skydiver to make the wrong decision, even though both the predicted and true probabilities indicate the same outcome (survive) is more likely than the alternative (die).

Improve this answer
$\endgroup$
1
  • $\begingroup$ +1. OP can find more information on examples like this by searching “predictive calibration” or simply “calibration”. $\endgroup$
    – Demetri Pananos
    Aug 14 at 13:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.