Consider the random variables $Y,X$. I believe that we can always write $$ Y=E(Y|X)+\epsilon $$ with $E(\epsilon|X)=0$.

Question: Is the above true regardless whether $Y$ is a discrete or continuous random variable?

My thoughts: I believe that the answer is yes.

For example, let $Z$ be a discrete random variable and $Y\equiv \mathbb{1}\{Z=3\}$, where $\mathbb{1}\{\cdots\}$ is the indicator function taking value $1$ if the condition inside is satisfied and zero otherwise. We can write $$ \mathbb{1}\{Z=3\}= E(\mathbb{1}\{Z=3\}| X)+ \epsilon \quad E(\epsilon|X)=0 $$ that is $$ \mathbb{1}\{Z=3\}= Pr(Z=3|X)+ \epsilon \quad E(\epsilon|X)=0 $$

  • 2
    $\begingroup$ Sure you can--but this decomposition is not always meaningful or mathematically helpful. For instance, let $X$ have a Gamma distribution and $Y$ have a Poisson$(X)$ distribution. The possible values $\epsilon$ can take on (all with positive probability) are the possible values of $Y-E(Y\mid X) = 0-X, 1-X, 2-X, \ldots.$ When $X$ is not integral (and it has zero chance of being an integer) this isn't a particularly nice set of values. Even worse, these sets differ for all $X,$ meaning there's not a lot of commonality among the $\epsilon.$ $\endgroup$ – whuber Feb 11 at 18:59

Yes this always possible, as long as the expectations exist:

$$E(Y|X) = E(Y|X).$$ $$E(Y|X) + Y= E(Y|X) + Y.$$ $$Y= E(Y|X) + Y - E(Y|X).$$ $$Y= E(Y|X) + \epsilon.$$

We then have $E(\epsilon) = E(Y - E(Y|X)) = E(Y) - E(E(Y|X)) = E(Y) - E(Y) = 0$ by linearity of expectations and the law of iterated expectations.

Note that in general and especially when $Y$ is discrete, $E(Y|X)$ will be non-linear in $X$.

  • $\begingroup$ Thanks. Is $E(\epsilon|X)=0$ by construction? $\endgroup$ – TEX Feb 11 at 16:59
  • $\begingroup$ Well, as I show it follows from linearity of expectations and the LIE, which follow from probability axioms and further definitions, so yes, in this sense (and I believe this is the "common sense"), it holds by constructing/defining $E(\epsilon|X)$ as $Y - E(Y|X)$. $\endgroup$ – Julian Schuessler Feb 11 at 17:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.