Expected square error formula I am going through this blog post to understand the Variance - Bias Tradeoff. In this, the author mentions:
Err(x) = E[(Y - f'(x))^2]
I understand this. But then, she decomposes it as:
Err(x) = (E([f'(x)] - f(x))^2 + E[(f(x) - E[f'(x)])^2]
I do not understand where this came from. I am aware that E(X^2) = Var(X) + (E[X])^2. But how does that lead to the above expression?
 A: You've a typo. In the second summand, it should be $\hat f(x)$. Substitute $y=f(x)+\epsilon$, and assuming $x$ is constant:
$$\begin{align}\mathbb E[(y-\hat f(x))^2]&=\mathbb E[(f(x)+\epsilon-\hat f(x))^2]\\&=\mathbb E[(f(x)-\hat f(x))^2]+\mathbb E[\epsilon^2]+\overbrace{2\mathbb E[\epsilon (f(x)-\hat f(x))]}^0\\&=\mathbb E[(f(x)-\mathbb E[\hat f(x)]-(\hat f(x)-\mathbb E[\hat f(x)]))^2]+\sigma_\epsilon^2\\&=\mathbb E[(f(x)-\mathbb E[\hat f (x)])^2]+\mathbb E[(\hat f(x)-\mathbb E[\hat f(x)])^2]\\&\ \ \ \ \ \ \ \ -\overbrace{2\mathbb E[(f(x)-\mathbb E[\hat f(x)])(\hat f(x)-\mathbb E[\hat f(x)])]}^{0}+\sigma_\epsilon^2\\&=\mathbb E[(f(x)-\mathbb E[\hat f (x)])^2]+\mathbb E[(\hat f(x)-\mathbb E[\hat f(x)])^2]+\sigma_\epsilon^2\\&= (\mathbb E[\hat f (x)]-f(x))^2+\mathbb E[(\hat f(x)-\mathbb E[\hat f(x)])^2]+\sigma_\epsilon^2
\end{align}$$
The first $0$ is simple (with the typical assumption that error is orthogonal to predicted $y$, i.e. $\hat y=\hat f(x)$):
$$\mathbb E[\epsilon (f(x)-\hat f(x))]=\mathbb E[\epsilon]\mathbb E[f(x)]-\overbrace{\mathbb E[\epsilon\hat f(x)]}^0=0$$
The second one is:
$$\begin{align}\mathbb E[(f(x)-\mathbb E[\hat f(x)])(\hat f(x)-\mathbb E[\hat f(x)])]&=\mathbb E[f(x)-\mathbb E[\hat f(x)]]\mathbb E[\hat f(x)-\mathbb E[\hat f(x)]]\\&=(f(x)-\mathbb E[\hat f(x)])\underbrace{(\mathbb E[\hat f(x)]-\mathbb E[\hat f(x)])}_0\\&=0\end{align}$$
