Suppose that $X$ and $Y$ have an uknown joint distribution $f_{XY}$.

How can I formally demostrate that it always exists a unique decomposition of the form : $$ Y = E[Y|X] +\epsilon $$

without assuming any explicity form of $f_{XY}$?


This follows from the linearity of expectation and law of total expectation

$X$ and $Y$ have an unknown joint distribution $F_{XY}$, and the distribution of $Y\mid X$ is some unknown $F_{Y|X}$. Suppose the mean of $Y|X$ is $\mu(X)$.

Consider the random variable $\epsilon = Y - \mu(X)$. Then $\epsilon$ is a mean 0 random variable. To see this, \begin{align*} E(\epsilon)& = E(Y - \mu(X))\\ & = E(E(Y - \mu(X) \mid X))\\ & = E\left[ E(Y\mid X) - \mu(X) \right]\\ & = E(0)\\ & = 0\,. \end{align*}

Thus, we can always write

$$Y = \mu(X) + \epsilon = E(Y\mid X) + \epsilon \,.$$


Assume that there exists a $\delta(X)$ and $\eta$, such that

$$Y = \delta(x) + \eta. $$

Uniqueness holds under two constraints.

  1. $\eta$ is independent of $X$
  2. $E(\eta) = 0$

Taking expectation with respect to $F_{Y|X}$, \begin{align*} E(Y\mid X) & = E( \delta(X) \mid X) + E(\eta \mid X)\\ \Rightarrow \mu(X) & = \delta(X) + 0\,. \end{align*}

Thus, we obtain that $\delta(X) = \mu(X)$, which implies $\eta = \epsilon$.

Note that the two constraints are crucial.

  • $\begingroup$ Thanks for the help. I'm ok with the demostration that if you define $\epsilon$ as $ Y - \mu (X)$ , the mean of this random variable is $0$ but how does it implies that there's an unique decomposition of $Y$ ? . If you define $\epsilon = \lambda (Y - \mu (X) ) $ you can actually show likewise that $E [\epsilon] =0$ . $\endgroup$ – R.Lac Apr 24 '18 at 9:27
  • $\begingroup$ @R.Lac, yes but then you obtain the equation $\lambda Y = \lambda \mu(X) + \epsilon$, which is not what you need. $\endgroup$ – Greenparker Apr 24 '18 at 9:30
  • $\begingroup$ Sorry, maybe i'm overthinking on it but I miss the point. If I obtain the equation as you shown, is possible to write a decomposition of $Y$ of the form $ Y = E[Y|X] + \frac{\epsilon}{\lambda} $ so if you recall $\frac{\epsilon}{\lambda} = \eta $ , $Y$ has infinite many decomposition of that form. $\endgroup$ – R.Lac Apr 24 '18 at 9:37
  • 1
    $\begingroup$ @R.Lac I realized this needed more explanation. I have added a discussion on uniqueness of such a decomposition. The key aspect is the constraints imposed on the decomposition. $\endgroup$ – Greenparker Apr 24 '18 at 9:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.