From Wikipedia the law of total expectations provides that:

Eq. 1: $$ E[E[\epsilon|X]] = E[\epsilon] $$

I have asked a question here, and the answer provided the law of total expectation as:

Eq 2: $$ E[X^T \epsilon]=E[X^TE[\epsilon | X ]] $$

Provided that $X$ is a stochastic variable, and thus cannot be taken out from the expectation operator, how can I go from eq. (1) to eq. (2)?

  • $\begingroup$ Could you explain what you mean by "cannot be taken out from"? By definition, $E[\epsilon\mid X]$ is a random variable, whence $X^\prime E[\epsilon\mid X]$ is a random variable and therefore its expectation is a number. $\endgroup$
    – whuber
    Mar 22 at 15:18
  • $\begingroup$ If this is regression, then we don't treat $X$ as stochastic. $\endgroup$
    – AdamO
    Mar 22 at 15:54
  • $\begingroup$ @AdamO in the linked answer, there is written that it covers the "case in which the regressors are also treated as random variables" $\endgroup$ Mar 22 at 16:03

There's a few things to clear up here. First off, it's probably helpful to rewrite the law of total expectations without using your variables, since you may give them linear regression interpretations. Given any rvs $W,V$, the law of total expectations says that $$E[W] = E[E[W|V]].$$

So what you "feed" as $W$ and $V$ in the above will obviously change the interpretation. Eq 1 takes $W = \epsilon$ and $V = X$, whereas Eq 2 takes $W = X^T\epsilon$ and $V =X$ and uses the property of conditional expectations that $E[VW|V] = VE[W|V]$ to the get the RHS of Eq 2. If $X,\epsilon$ were just any random variables, theres no reason to expect them to be equal.

As it turns out, you asked the linked question in the context of a linear regression, where you assume $E[\epsilon|X]=0$. In this case, then they are indeed equal, because $$E[\epsilon] = E[E[\epsilon|X]] = E[0]=0$$ and $$E[X^T\epsilon] = E[E[X^T\epsilon|X] = E[X^TE[\epsilon|X]] = E[X^T0] = 0.$$

So in this case, you can add whatever rv $W$ into $E[WE[\epsilon|X]]$ and you will get that it is $0$ because $E[\epsilon|X] = 0$! But this shouldn't amaze you, because you just assumed that it was $0$, and it wasn't anything you derived.

  • $\begingroup$ Thanks very much $\endgroup$ Mar 22 at 16:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.