# Quadratic loss function implying conditional expectation

I am reading Bishop's pattern recognition book. In the decision theory part he first derives that using a quadratic loss function implies that our estimate $y(x)$ should be the conditional expectation $\mathbb{E}[t|x]$. Then, armed with this knowledge he wants to show a simpler proof: \begin{align*}L(t, y(x)) &= (y(x) - t)^2 \\&= (y(x) - \mathbb{E}[t|x] + \mathbb{E}[t|x] - t)^2 \\&= (y - \mathbb{E}[t|x])^2 + 2(y(x) - \mathbb{E}[t|x])(\mathbb{E}[t|x] - t) + (\mathbb{E}[t|x] - t)^2\end{align*}

So far so good but now he says "performing the integral over t, we see that the cross-term vanishes and we obtain an expression for the loss function in the form"

$$\mathbb{E}[L] = \int (y - \mathbb{E}[t|x])^2 p(x) dx + \int (\mathbb{E}[t|x] - t)^2 p(x) dx$$

Now, how does the cross-term vanish exactly? Also how can the above expression still contain $t$ (he just integrated over it, so he probably means $(\mathbb{E}[t|x] - \mathbb{E}[t])^2$ ?).

This is on page 47 of Bishop: Pattern Recognition And Machine Learning

First, taking the expectation in both $t$ and $x$ leads to \begin{align*}\overbrace{\mathbb{E}[L(t, y(x))]}^\text{both $t$ and $x$ are rvs} &= \mathbb{E}[(y(x) - t)^2] \\&= \mathbb{E}[(y(x) - \mathbb{E}[t|x] + \mathbb{E}[t|x] - t)^2] \\&= \mathbb{E}[(y - \mathbb{E}[t|x])^2] + \mathbb{E}[2(y(x) - \mathbb{E}[t|x])(\mathbb{E}[t|x] - t)] + \mathbb{E}[(\mathbb{E}[t|x] - t)^2]\\ &= \mathbb{E}[(y - \mathbb{E}[t|x])^2] + 2\overbrace{\mathbb{E}[\underbrace{(y(x) - \mathbb{E}[t|x])}_\text{function of $x$}\underbrace{\mathbb{E}[(\mathbb{E}[t|x] - t)|x]}_\text{expectation over $t$ given $x$}]}^\text{expectation over $x$} + \mathbb{E}[(\mathbb{E}[t|x] - t)^2]\\&= \mathbb{E}[(y - \mathbb{E}[t|x])^2] + 2\mathbb{E}[(y(x) - \mathbb{E}[t|x])\underbrace{(\mathbb{E}[t|x]- \mathbb{E}[t|x])}_{\mathbb{E}[\mathbb{E}[t|x]|x]= \mathbb{E}[t|x]}] + \underbrace{\mathbb{E}[\mathbb{E}[(\mathbb{E}[t|x] - t)^2|x]]}_\text{double expectation}\\&= \mathbb{E}[(y - \mathbb{E}[t|x])^2] + 2\mathbb{E}[(y(x) - \mathbb{E}[t|x])\times 0] + \mathbb{E}[\mathrm{var}[t|x]]\\&= \underbrace{\mathbb{E}[(y - \mathbb{E}[t|x])^2]}_\text{squared bias} + \underbrace{\mathbb{E}[\mathrm{var}[t|x]]}_\text{independent of $y$}\end{align*} [where the double expectation on line 4 means first integrating according to $p(t|x)$ and then according to $p(x)$]. Hence, taking $y=\mathbb{E}[t|x]$ minimises the above.