# Calibration Expectation Decompostion

I am reading a "Calibrated Structured Prediction by Kuleshov and Liang" link.

Calibration and sharpness. Given a forecaster $$F : X → [0, 1]$$, define $$T(x) = \mathbb{E}[y| F(x)]$$ to be the true probability of $$y = 1$$ given a that $$x$$ received a forecast $$F(x)$$. We can use $$T$$ to decompose the $$l_2$$ prediction loss as follows: $$\mathbb{E}[(y-F(x))^2] = \mathbb{E}[(y-T(x))^2] + \mathbb{E}[(T(x)-F(x))^2]=\text{var}[y]-\text{var}[T(x)]+ \mathbb{E}[(T(x)-F(x))^2]$$ The first equality follows because $$y−T(x)$$ has expectation $$0$$ conditioned on $$F(x)$$, and the second equality follows from the variance decomposition of $$y$$ onto $$F(x)$$.

Trying to prove the above. What I have got so far. \begin{align} \mathbb{E}[(y-F(x))^2] & = \mathbb{E}[(y-T(x)+T(x)-F(x))^2] \\\\ & = \mathbb{E}[(y-T(x))^2]+ \mathbb{E}[(T(x)-F(x))^2]+ 2\mathbb{E}[(y-T(x))(T(x)-F(x))]\\ \end{align}

For the last term in above using law of total expectation, conditioning on $$F(x)$$:

$$\begin{equation*} \mathbb{E}[(y-T(x))(T(x)-F(x))]=\mathbb{E}[\mathbb{E}[(y-T(x))(T(x)-F(x))\vert F(x)]] \end{equation*}$$

The second parenthesis is a function of $$F(x)$$ so can come out of conditioning. \begin{align*} \mathbb{E}[(y-T(x))(T(x)-F(x))]&=\mathbb{E}[(T(x)-F(x))\mathbb{E}[(y-T(x))\vert F(x)]]\\\\ & = \mathbb{E}[(T(x)-F(x))(T(x)-T(x))]=0 \end{align*} Which used the definition of $$T(x)$$.

So far we have: $$$$\mathbb{E}[(y-F(x))^2]=\mathbb{E}[(y-T(x))^2] + \mathbb{E}[(T(x)-F(x))^2]$$$$

However, I can not close the derivation by proving: $$\begin{equation*} \mathbb{E}[(y-T(x))^2] =\text{var}[y]-\text{var}[T(x)] \end{equation*}$$

## 1 Answer

I think the key insight is that from the definition of conditional expectation it follows that $$\mathbb{E} [ y \mathbb{E} (y | F(x)) ] = \mathbb{E}[ \mathbb{E}(Y | F(x))^2 ]$$ Hence, \begin{align*} \mathbb{E}[(y-T(x))^2 ] &= \mathbb{E}[y^2] - 2 \mathbb{E} [ y \mathbb{E} (y | F(x)) ] + \mathbb{E}[\mathbb{E}(Y | F(x))^2] \\ &= \mathbb{E}[y^2] - \mathbb{E}[\mathbb{E}(Y | F(x))^2] \\ &= \mathbb{E}[y^2] - (\mathbb{E}[y])^2 - \left\{ \mathbb{E}[\mathbb{E}(Y | F(x))^2] - (\mathbb{E}[y])^2 \right\} \\ &= \mathrm{var}[y] - \mathrm{var}[\mathbb{E}(y | F(x))] , \end{align*} where the last equality follows from the law of total expectation.