# Why the law of total expectations can be written as $E[X^T \epsilon]=E[X^TE[\epsilon | X ]]$

From Wikipedia the law of total expectations provides that:

Eq. 1: $$E[E[\epsilon|X]] = E[\epsilon]$$

I have asked a question here, and the answer provided the law of total expectation as:

Eq 2: $$E[X^T \epsilon]=E[X^TE[\epsilon | X ]]$$

Provided that $$X$$ is a stochastic variable, and thus cannot be taken out from the expectation operator, how can I go from eq. (1) to eq. (2)?

• Could you explain what you mean by "cannot be taken out from"? By definition, $E[\epsilon\mid X]$ is a random variable, whence $X^\prime E[\epsilon\mid X]$ is a random variable and therefore its expectation is a number.
– whuber
Mar 22 at 15:18
• If this is regression, then we don't treat $X$ as stochastic. Mar 22 at 15:54
• @AdamO in the linked answer, there is written that it covers the "case in which the regressors are also treated as random variables" Mar 22 at 16:03

There's a few things to clear up here. First off, it's probably helpful to rewrite the law of total expectations without using your variables, since you may give them linear regression interpretations. Given any rvs $$W,V$$, the law of total expectations says that $$E[W] = E[E[W|V]].$$
So what you "feed" as $$W$$ and $$V$$ in the above will obviously change the interpretation. Eq 1 takes $$W = \epsilon$$ and $$V = X$$, whereas Eq 2 takes $$W = X^T\epsilon$$ and $$V =X$$ and uses the property of conditional expectations that $$E[VW|V] = VE[W|V]$$ to the get the RHS of Eq 2. If $$X,\epsilon$$ were just any random variables, theres no reason to expect them to be equal.
As it turns out, you asked the linked question in the context of a linear regression, where you assume $$E[\epsilon|X]=0$$. In this case, then they are indeed equal, because $$E[\epsilon] = E[E[\epsilon|X]] = E[0]=0$$ and $$E[X^T\epsilon] = E[E[X^T\epsilon|X] = E[X^TE[\epsilon|X]] = E[X^T0] = 0.$$
So in this case, you can add whatever rv $$W$$ into $$E[WE[\epsilon|X]]$$ and you will get that it is $$0$$ because $$E[\epsilon|X] = 0$$! But this shouldn't amaze you, because you just assumed that it was $$0$$, and it wasn't anything you derived.