Decomposition of average squared bias (in Elements of Statistical Learning) I can't figure out how formula 7.14 on page 224 of The Elements of Statistical Learning is derived. Can anyone help me figure it out?  
$$\textrm{Average squared bias} = \textrm{Average}[\textrm{model bias}]^2 + \textrm{Average}[\textrm{estimation bias}]^2$$

 A: The result is basically due to the property of best linear estimator. Note that we don't assume $f(X)$ is linear here. Nevertheless we can find the linear predictor that approximates $f$ the best. 
Recall the definition of $\beta_*$: $\beta_{*} = \arg\min_\beta E{[(f(X) - X^T \beta)^2]}$. We can derive the theoretical estimator for $\beta_*$:
\begin{align*}
    g(\beta)
    &= E[(f(X) - X^T \beta)^2] = E [f^2(X)] - 2\beta^T E[Xf(X)] + \beta^T E[XX^T]\beta \\
    &\implies \frac{\partial{g(\beta)}}{\partial{\beta}} = -2 E{[Xf(X)]} + 2 E[XX^T]\beta = 0 \\
    &\implies \beta_{*} = E[X X^T]^{-1}E[X f(X)],
\end{align*} where we have assumed $E[X X^T]$ is invertible. I call it theoretical estimator as we never know (in real world scenarios anyways) the marginal distribution of X, or $P(X)$, so we won't know those expectations. You should still recall the resemblance of this estimator to the ordinaory least square estimator (if you replace $f$ with $y$, then the OLS estimator is the plugin equivalent estimator. at the end I show they are the same for estimating the value of $\beta_*$), which basically tells us another way of deriving the OLS estimator (by large number theory). 
The L.H.S of (7.14) can be expanded as:
\begin{align*}
    E_{x_0}[f(x_0) - E{\hat{f}_\alpha (x_0)}]^2 &= E_{x_0}[f(x_0) -x_0^T\beta_{*}+ x_0^T\beta_{*} - E{\hat{f}_\alpha (x_0)}]^2 \\
    &= E_{x_0}[f(x_0) - x_0^T\beta_{*}]^2 + E_{x_0}[ x_0^T\beta_{*} - E{\hat{f}_\alpha (x_0)}]^2 \\ &\;\;+ 2 E_{x_0}[(f(x_0) - x_0^T\beta_{*})(x_0^T\beta_{*}-E{\hat{f}_\alpha (x_0)})]. 
\end{align*}
To show (7.14), one only needs to show the third term is zero, i.e. 
$$E_{x_0}[(f(x_0) - x_0^T\beta_{*})(x_0^T\beta_{*}-E{\hat{f}_\alpha (x_0)})] = 0, $$
where the L.H.S equals
\begin{align*}
  LHS = E_{x_0}[(f(x_0) - x_0^T\beta_{*})x_0^T\beta_{*}] - E_{x_0}[(f(x_0) - x_0^T\beta_{*})E{\hat{f}_\alpha (x_0)})]
\end{align*}
The first term (for convenience, I have omitted $x_0$ and replace it with $x$):
\begin{align}
    &E{[(f(x) - x^T\beta_{*})x^T\beta_{*}]} = E{[f(x)x^T\beta_*]}- E{[(x^T\beta_*)^2]} \\
    &= E[f(x)x^T]\beta_* - \left(Var{[x^T\beta_*]} + (E{[x^T\beta_*]})^2\right) \\
    &= E[f(x)x^T]\beta_* - \left( \beta_*^T Var{[x]} \beta_* + (\beta_* ^T E[x])^2\right) \\
    &= E[f(x)x^T]\beta_* - \left( \beta_*^T (E[xx^T] - E[x]E[x]^T) \beta_* + (\beta_* ^T E[x])^2\right)  \\
    &= E[f(x)x^T]\beta_* -  E{[f(x)x^T]}E[xx^T]^{-1} E[xx^T]\beta_*  + \beta_*^TE[x]E[x]^T \beta_*\\  &\;\;- \beta_*^TE[x]E[x]^T \beta_* \\
    &= 0,
\end{align}
where we have used the variance identity $Var{[z]} = E{[zz^T]} - E{[z]}E{[z]}^T$ twice for both the second and forth step; we have substituted $\beta_*^T$ in the second last line and all the other steps follow due to standard expectation/variance properties. In particular, $\beta_*$ is a constant vector w.r.t the expectation, as it is independent from where $x$ (or $x_0$) is measured.
The second term \begin{align}
    E{[(f(x) - x^T\beta_{*})E{\hat{f}_\alpha (x)}]} &= E{[(f(x) - x^T\beta_{*}) E{[x^T\hat{\beta}_\alpha]}]} \\
    &= E{[E{[\hat{\beta}_\alpha}^T]x (f(x) - x^T\beta_{*})]} \\
    &= E{\hat{\beta}_\alpha}^TE{[x f(x) - x x^T\beta_*]} \\
    &=E{\hat{\beta}_\alpha}^T\left( E{[x f(x)]} - E[xx^T] E[xx^T]^{-1}E{[xf(x)]}\right)\\
    &=0,
\end{align}
where the second equality holds because $E{\hat{f}_\alpha (x)}$ is a point-wise expectation where the randomness arises from the training data $y$, so $x$ is fixed; the third equality holds as $E{\hat{\beta}_\alpha}$ is independent from where $x$ ($x_0$) is predicted so it's a constant w.r.t the outside expectation. 
Combining the above results, the sum of these two terms is zero, which shows eq.(7.14). 
Although not related to the question, it is worth noting that $ f(X) = E[Y|X]$, i.e. $f(X)$ is the optimal regression function, as 
$$ f(X) = E{[f(X) +\varepsilon |X]}  = E[Y|X].$$ Therefore,
\begin{align}
    \beta_{*} &= E[XX^T]^{-1}E{[Xf(X)]}  = E[XX^T]^{-1}E{[XE[Y|X]]} \\
    &= E[XX^T]^{-1}E[E[XY|X]] \\
    &= E[XX^T]^{-1}E[XY],
\end{align} if we recall the last estimator is the best linear estimator, the above equation basically tells us, using the optimal regression function $f(x)$ or the noisy version, $y$ is the same as far as the point estimator is the concern. Of course, the estimator with $f$ will have better property/efficiency as it will lead to smaller variance, which can be easily seen from that fact $y$ introduces extra error, or variance.
A: Yes, this isn't the usual bias-variance decomposition, rather it's a further decomposition of the bias.
Someone else has already answered this.
