# What is the proper statistical way to meausre the confidence of a probability distribution?

I have a 3-class probability distribution produced by a classification model. For example:

$$y = [0.1, 0.7, 0.2]$$

I want to plot a simple measure of confidence for a set of predictions.

What I've done is treat $$y$$ as a vector and take its magnitude. Chance is the lowest confidence with a magnitude of:

$$|y_{min}| = |[0.33, 0.33, 0.33]| = 0.576773$$

A 100% confident assignment has a magnitude of 1:

$$|y_{max}| = |[1, 0, 0]| = 1.0$$

Then I simply rescale $$y$$ to a $$[0, 1]$$ range to produce the confidence

$$\text{confidence}(y) = \frac{|y| - |y_{min}|}{|y_{max}|-|y_{min}|}$$

My questions:

• Is there anything problematic with that measure of confidence?
• Is there another method that is more applicable to this case?
• Would variance work?
– Dave
Commented May 21, 2021 at 2:01
• Good idea, variance can work in the same way. It produces a different scale of the confidence. Is there a reason why one method vs. another would be preferred? Commented May 21, 2021 at 2:31
• People are familiar with variance.
– Dave
Commented May 21, 2021 at 3:13
• The variance of what? We have probabilities, but no values - there is no variance to compute here. Commented May 21, 2021 at 3:33
• Entropy is likely to be what your looking for. Low entropy (zero) corresponds to your confidence = 1 case. The uniform case will have the highest entropy. Commented May 21, 2021 at 4:00