Is this an application of Frisch-Waugh-Lovell theorem?

Question

Firstly, the regression of Y on $$X_1, X_2, X_3, X_4$$ yields: $$Y=\beta_0+\beta_1X_1+\beta_2X_2+\beta_3X_3+\beta_4X_4+e (1)$$ Then, regress Y on $$X_2, X_3, X_4$$ $$Y=\beta'_0+\beta'_2X_2+\beta'_3X_3+\beta'_4X_4+e'(2) $$ Lastly, regress $$X_1$$ on $$X_2, X_3, X_4$$ $$X_1=\beta''_0+\beta''_2X_2+\beta''_3X_3+\beta''_4X_4+e'' (3)$$

Now I regress e' on e'' $$e'= \alpha_1+\alpha_2e''+\epsilon$$

Is there a simple method to prove that $$\alpha_2=\beta_2$$ Can I plug (3) into (1) (which yields equation (4) ) and then identity the residuals of (2) and (4) ?

p/s: I'm sorry but can anyone tell me how to add comment? I try the button "add comment" but I still can't. I'm new here.

Answer 1

I will give you an aswer considering a regression based of $Y$ on $X_1$ and $X_2$ only, since it is clearer and it simplifies the algebra. It does not change anything in terms of intuition or results. Note that I am assuming that the intercept is part of $X_1$, and that you regress it only once (which is all that matters, you could also assume that the vector of $1$ is part of $X_2$.

---

First we regress $Y = X_2 \beta_2 + \epsilon$ (so, without $X_1$)

The coefficient $\hat{\beta}_2 = (X_2'X_2)^{-1}X_2'Y$

Therefore the residual $e= Y -X_2(X_2'X_2)^{-1}X_2'Y = [\mathbf{I} -X_2(X_2'X_2)^{-1}X_2']Y = M_2 Y$, where $M_2$ is the orthogonal component to $X_2$

---

Second, we regress the auxiliary $X_1 = X_2 \gamma_2 + \epsilon$

Following the same steps you will find that $e_{aux} = M_2X_1$

---

As a final step, we regress $e = e_{aux}\delta$ and we want to show that $\hat{\delta} = \hat{\alpha}_2$, where $\alpha_2$ is the coefficient from the "full" regression $Y = X_1\alpha_1 + X_2\alpha_2 + \epsilon$

Substituting from the first two parts:

$M_2Y = M_2X_1\delta +\epsilon$

Therefore $\hat{\delta} = (X_1'M_2'M_2X_1)^{-1}X_1'M_2'M_2Y = (X_1'M_2X_1)^{-1}X_1'M_2Y$ since $M_2$ is idempotent.

---

Finally, what is the FWL theorem telling you? That running the whole regression or doing this steps gives the same coefficient, since we are removing from each variable the part already explained by the others.

How to see that it is actually the same? Set the normal equations for the full regression, compute $\hat{\alpha_1}$ and $\hat{\alpha_2}$, plug one of the two in the expression of the other. You will eventually obtain the same matrix result I show for $\delta$.