Proof of the Linear Conditional Expectation Function theorem

Question

I'm currently trying to re-teach myself some econometrics (last studied about 10 years ago) by reading Causal Inference, the mixtape and the classic Mostly Harmless Econometrics.

I'm currently running through the three arguments as to why we should think that linear regression is a sensible thing to do. The first of these is presented, in both texts, as the Linear Conditional Expectation Function (CEF) theorem.

The Linear CEF theorem states that if the CEF is linear, then it is the same as the population regression.

The proof runs as follows:

First lets assume that the CEF is linear, so that we have $E(y_i\vert X)=\beta^* X_i$.

The proof the asks us to recall that by the CEF decomposition we have: $$E(X_i(y_i-E(y_i\vert X_i)))=E(X_i(y_i-\beta^*X_i))=0$$

My question is, where does this last line come from? As far as I'm concerned the CEF decomposition tells us that we have $y_i=E(y_i\vert X_i) + \epsilon_i$, where $\epsilon_i$ is mean independent of $X_i$ and uncorrelated with any function of $X_i$.

I'm really not clear how that gives us the second line of the above proof.

Answer 1

$$E(X_i(y_i-E(y_i\vert X_i)))=E(X_iy_i)-E(X_iE(y_i|X_i))=E(E(X_iy_i|X_i))-E(X_iE(y_i|X_i))=E(X_iE(y_i|X_i))-E(X_iE(y_i|X_i))=0$$

Where the first equality makes use of the linearity of the expectation and the second equality makes use of the Law of Iterated Expectations.

Alternatively, it follows from the fact which you note ("$\varepsilon_i$ is mean independent of $X_i$ and uncorrelated with any function of $X_i$" ): $$E(X_i(y_i-E(y_i|X_i)))=E(X_i \varepsilon_i)=0$$

Answer 2

Note that we start by assuming that the

CEF is linear, so that we have $\mathbb{E}[y_i | X] = \beta^*X_i$

So, for a given $x_i$, the conditional mean will just be a scalar multiple of $x_i$, in this case the scalar is the coefficient $\beta^*$.

Then you mention that

[...] the CEF decomposition tells us that we have $y_i = \mathbb{E}[y_i | X_i] + \epsilon_i$

You can see this in one way as that the response for a particular individual is just their mean given the other covariates, $\mathbb{E}[y_i | X_i]$, plus the unique deviation from the mean that individual has, that is $\epsilon_i$.

You can re-order this equation to show that what the CEF decomposition is doing here is defining what the error term $\epsilon_i$ is, namely:

$\epsilon_i = y_i - \mathbb{E}[y_i | X_i]$

Simply, that the difference between the observed response $y_i$ and its conditional mean given the covariates is the error term $\epsilon_i$. Hence we can always replace one with the other.

Lastly, note that you also mention

$\epsilon_i$ is mean independent of $X_i$

, hence we have all our ingredients to show that:

\begin{align} &\mathbb{E}[X_i(\epsilon_i)] = 0 \tag{By uncorrelatedness$^1$} \\ &\mathbb{E}[X_i(y_i - \mathbb{E}[y_i|X_i])] = 0 \tag{By definition of $\epsilon_i$} \\ &\mathbb{E}[X_i(y_i - \beta^* X_i])] = 0 \tag{By linearity of CEF} \\ \end{align}

---

1. Uncorrelatedness is implied by mean independence. Here for simplicity we are assuming that the variables are centered, so that $\mathbb{E}[XY] = \mathbb{E}[(X - \mathbb{E}[X])(Y - \mathbb{E}[Y])] = \operatorname{Cov}(X, Y)$