2
$\begingroup$

I have a model $Y = f(X) + \epsilon$

where $\epsilon$ is independent of $X$ and $\mathbb{E}[\epsilon]=0, \mathbb{E}\left[\epsilon^2\right]=\sigma^2$.

Show that $$ f(X)=\mathbb{E}[Y \mid X] $$

This is my attempt at the proof:

$$ \begin{aligned} & E[Y \mid X] \\ &= E[f(X)+\varepsilon \mid X] \\ &= E[f(X) \mid X]+E[\varepsilon \mid X] \\ &= E[f(X) \mid X]+E[\varepsilon] \\ &= E[f(X) \mid X]+0 \\ &= E_{Y \mid X}[f(X) \mid X] \quad\quad\quad \quad \quad\ \ (5) \\ &= \sum_{y \mid x} f(X) P(Y|X=y| x) \quad\quad(6) \\ &= f(X) \sum_{y \mid x} P(Y|X=y| x) \quad\quad(7)\\ &= f(X) \end{aligned} $$

I am a bit iffy about steps (5) to (6) to (7), please help. Thank you!

Improve this question
$\endgroup$
2
  • 5
    $\begingroup$ You were done before you reached step (5). In fact, at that point $Y$ no longer appears anywhere. You can't just stick $Y$ back in, either, because it involves $\epsilon,$ which also has vanished. A basic property of conditional expectations -- usually proven as soon as conditional expectation is defined -- is that $E[f(X)\mid X] = f(X).$ This is called "taking out what is known." $\endgroup$
    – whuber ♦
    Commented Sep 30, 2022 at 22:18
  • $\begingroup$ Thank you! understood $\endgroup$
    – Cabbage Roll
    Commented Oct 3, 2022 at 1:26

1 Answer 1

Reset to default
3
$\begingroup$

$\mathbb{E}[Y|X] = \mathbb{E}[f(X)+\epsilon|X] = \mathbb{E}[f(X)|X]+\mathbb{E}[\epsilon|X] = \mathbb{E}[f(X)|X]+\mathbb{E}[\epsilon] = f(X)+0$

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ I think it is worth emphasizing that it is merely assumed that the conditional expectation of the true error (conditional on X) is zero, and that without this assumption, the theorem is not valid. $\endgroup$
    – Peter Leopold
    Commented Oct 1, 2022 at 5:14
  • 1
    $\begingroup$ @PeterLeopold The OP asssumes that X and $\epsilon$ are independent. $\endgroup$
    – John Madden
    Commented Oct 1, 2022 at 13:03
  • $\begingroup$ Thanks both, understood $\endgroup$
    – Cabbage Roll
    Commented Oct 3, 2022 at 1:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.