How to evaluate the quality of the probability distribution output of a classifier?

Question

In a classification problem, I have trained a neural network which outputs class probabilities for a given input. For a new input, I now want to evaluate the "quality" of the neural network's classification score. I'm not looking to evaluate the quality of the overall neural network, but I want the quality for that specific input. What are some examples of ways to measure this?

A simple way would be to just give a 1 or 0 depending on whether the neural network classified the input correctly. Or to get a continuous number, you could take the value that the neural network outputs for the ground-truth class. However, this does not consider all the other classes. It makes sense that if the ground-truth is class 1, then if the neural network outputs (class1=0.6, class2=0.2, class3=0.2) then this is better than (class1=0.6, class2=0.3, class3=0.1) because in the second case, the neural network is less confident of the distinction between class1 and class2. This reminds me of entropy, although in this case, we do know the ground truth.

Any suggestions? Thanks.

Answer 1

You might want to use the logarithmic scoring rule. https://en.wikipedia.org/wiki/Scoring_rule

Not only the it is justifiable in general, in your example f(class1=0.6, class2=0.2, class3=0.2) = ln(0.6)+2*ln(0.2) = -3.729701449 f(class1=0.6, class2=0.3, class3=0.1)= ln(0.6)+ln(0.3)+ln(0.1) = -4.017383521

So you will get f(class1=0.6, class2=0.2, class3=0.2) > f(class1=0.6, class2=0.3, class3=0.1) as you wanted.

Answer 2

Assessing the quality of a multinomial classifier based on the modal class (the class that has the highest probability) is indeed sub-optimal. This results in the richer posterior class distribution (which is much more informative) to, essentially, go to waste.

The multinomial (generalised) use case for the Bayesian Information Reward (BIR) caters for that. In the Bayesian Information Reward paper, the authors highlight the problem you mention on Pg.3 (between equations 2b and 3) and continue to propose a solution.

$I_i^+ = 1-log\frac{pi}{p'i}$

$ I_i^- = 1-log\frac{1-pi}{1- p'i}$

$ BIR = \frac{\sum_i^k{I_i}}{k} $

where,

$pi$ is the predicted probability of class i

$p'i$ is the prior probability of class i

$k$ is the number of class labels

$I_i^+$ and $I_i^-$ are the rewards for correct and incorrect classifications respectively.

The $BIR$ for a single observation is calculated as the average of rewards across all labels.

In the context of Neural Networks training, Cross Entropy is commonly used. This is certainly better than training based on accuracy. However, note that $BIR$ is better than cross entropy since it takes into account class priors.