Why is binning in Expected Calibration Error done the way it is?

Question

I see the definition of expected calibration error being

$$\sum_{m=1}^{M}\frac{|B_m|}{n}|accuracy(B_m)-confidence(B_m)|$$

Where $B_m$ represents a outputs of the model that predicted class $m$ in a multi-classifcation problem. Then I read this nice article with an example of calculating it. So the binning is determined by what the model outputs as the max prob. So for example, if input 0 leads to a 90% prediction of "dog," that'll go into the "dog" bin. Once this is done the accuracy as in how many of them correctly estimated the class subtracted by the average confidence or predictive probability by the model is take. I'm a little bit confused about the binning part. What does binning the predictions in this way tell us exactly? Why not do the bins via the true classes? For example you take all the rows where the true class is $m$ then calculate the accuracy and confidence for class $m$?

Answer 1

The expected calibration error is (often) defined as:

$$\text{ECE}(f)=\mathbb{E}_{f(x)}\left[\left|f(X)-\mathbb{E}[Y \mid f(X)]\right|\right]$$

where $f(x) \in [0,1]$ is the NN output and $\mathbb{E}[Y \mid f(X)]=\mathbb{P}[Y=1 \mid f(X)]$ in the binary case. This is extended to multi-class problems by simply regarding every class as a binary classification objective and averaging over them (see https://arxiv.org/pdf/1909.10155.pdf).

Your formula is just an estimator of the expectation above:

$$\text{ECE}(f)=\mathbb{E}_{f(x)}\left[\left|f(X)-\mathbb{E}[Y \mid f(X)]\right|\right] \approx \sum_{m=1}^M \frac{|B_m|}{n}|conf(B_m)-acc(B_m)|$$

Because the $f(x)$ is continuous, we need to discretise it: Let us partition $[0,1]$ into $M$ intervals. Then we define $B_m$ to be the set data points that have predictions falling into the $m$th interval (see https://arxiv.org/abs/1706.04599 for more formal definition of $B_m$). So $m$ indicites the groupping based on confidence, not the predicted class itself.

Note: The $ECE$ is not a good metric as it has a lot of shortcomings. For instance, if $f(X) > 0.9$ for all data points, then the equal width binning of the $ECE$ is useless. Another issue is that the estimated $ECE$ tends to underestimate the true calibration error, which has been shown by multiple papers in different ways.

Answer 2

Expected Calibration Error indeed performs the binning rather naively based on equi-spaced fragments of the probability space. This is not great for problems with class imbalance, where for example most predictions will be heavily concentrated on the lower ends of the [0,1]-output range.

Adaptive Calibration Error (ACE) [1] and Test-based Calibration Error (TCE) [2] are two alternative calibration metrics that among others differ in the way that the binning procedure is performed.

The former simply bins by quantiles, such that each bin ends up having the same data volume. The latter defines the notion of optimal binning, under which a binning is said to be optimal when it minimises the expected estimation error weighted by the bin sizes, and propose an algorithm that can achieve such an optimal binning.

References:

[1] Nixon, J., Dusenberry, M. W., Zhang, L., Jerfel, G., & Tran, D. (2019). Measuring Calibration in Deep Learning. In CVPR workshops (Vol. 2, No. 7).

[2] Matsubara, T., Tax, N., Mudd, R., & Guy, I. (2023). TCE: A Test-Based Approach to Measuring Calibration Error. In: Proceedings of the Conference on Uncertainty in Artificial Intelligence, PMLR