Variance-bias trade-off equation

Question

If we consider the variance bias trade off equation, stated as:

$$\newcommand{\Var}{{\rm Var}} E(y_0-\hat{f}(x_0))^2=\Var(\hat{f}(x_0))+({\rm Bias}(\hat{f}(x_0)))^2+\Var(\varepsilon) $$

We assume that the model is $y=f(x)+\varepsilon$ and our test data is $(x_0,y_0)$.

If we assume we are using the correct model then the bias should be zero. Further, $E(\hat{f}(x_0))=y_0$ too, which leads to the left hand term of the original equation being equal to $\Var(\hat{f}(x_0))$. Putting all this together into the original equation leads us to find that $\Var(\varepsilon)=0$ which is clearly false by design.

Clearly there is something wrong with my understanding. Which step(s) of the above are therefore false?

Answer 1

The squared term on the left hand side should be inside the expectation. The equation should be: $$ E[(y_0 - \hat{f}(x_0))^2] = Var(\hat{f}(x_0)) + Bias(\hat{f}(x_0))^2 + Var(\epsilon)$$

Therefore you can't use linearity of expectation. If your equation was true, the LHS would actually be 0 for an unbiased $\hat{f}$.