3
$\begingroup$

On page 52 of The Elements of Statistical Learning edition 2, we are told to:

"Consider the prediction of the new response at input $x_0$:

$$Y_0 = f(x_0) + \epsilon_0$$

Then the expected prediction error of an estimate $\tilde{f}(x_0) = x_0^T\tilde{\beta} $ is

$$ \begin{align} E(Y_0 - \tilde{f}(x_0))^2 & = \sigma^2 + E(x_0^T\tilde{\beta} - f(x_0))^2 \\ & = \sigma^2 + MSE(\tilde{f}(x_0)) '' \end{align} $$

I have two questions:

(1) How do you decompose $E(Y_0 - \tilde{f}(x_0))^2$ into $\sigma^2 + E(x_0^T\tilde{\beta} - f(x_0))^2$ (how is it derived)?

(2) It seems like $E(Y_0 - \tilde{f}(x_0))^2$ and $MSE(\tilde{f}(x_0))$ should be the same thing. What is the difference between the two?

$\endgroup$

1 Answer 1

1
$\begingroup$

Using the assumptions of the data generating model defined on page 28, we have

\begin{aligned} \mathrm{E}[(Y_0-\tilde{f}(x_0))^2] &=\mathrm{E}[(f(x_0)+\epsilon-\tilde{f}(x_0))^2] \\ &= \mathrm{E}[(\tilde{f}(x_0)-f(x_0))^2] + 2\,\mathrm{E}[(\tilde{f}(x_0)-f(x_0))\cdot\epsilon] + \mathrm{E}[\epsilon^2] \\ &= \mathrm{E}[(\tilde{f}(x_0)-f(x_0))^2] + 2\,(\mathrm{E}[\tilde{f}(x_0)]-f(x_0))\cdot\underbrace{\mathrm{E}[\epsilon]}_{=0} + \sigma^2 \\ &= \mathrm{E}[(\tilde{f}(x_0)-f(x_0))^2] + \sigma^2 \\ &= \mathrm{MSE}[\tilde{f}(x_0)] + \sigma^2. \end{aligned}

$\endgroup$
1
  • $\begingroup$ Ok, so MSE doesn't take into account that $Y_0 = f(x_0) + \epsilon$, but $E[(Y_0 - \tilde{f}(x_0))^2]$ does. That would be the difference, correct? $\endgroup$
    – user239050
    Feb 26, 2019 at 3:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.