Utility of the Frisch-Waugh theorem

Question

I am supposed to teach the Frish Waugh theorem in econometrics, which I have not studied.

I have understood the maths behind it and I hope the idea too "the coefficient you get for a particular coefficient from a multiple linear model is equal to the coefficient of the simple regression model if you "eliminate" the influence of the other regressors". So the theoretical idea is kind of cool. (If I totally misunderstood I do welcome a correction)

But does it have some classical/practical usages ?

EDIT : I have accepted an answer, but am still willing to have new ones that bring other examples/applications.

Answer 1

Consider the fixed effects panel data model, also known as Least Squares Dummy Variables (LSDV) model.

$b_{LSDV}$ can be computed by directly applying OLS to the model $$y=X\beta+D\alpha+\epsilon,$$ where $D$ is a $NT\times N$ matrix of dummies and $\alpha$ represent the individual-specific fixed effects.

Another way to compute $b_{LSDV}$ is to apply the so called within transformation to the usual model in order to obtain a demeaned version of it, i.e. $$M_{[D]}y=M_{[D]}X\beta+M_{[D]}\epsilon.$$ Here, $M_{[D]}=I-D(D'D)^{-1}D'$, the residual maker matrix of a regression on $D$.

By the Frisch-Waugh-Lovell theorem, the two are equivalent, as FWL says that you can compute a subset of regression coefficients of a regression (here, $\hat\beta$) by

1. regressing $y$ on the other regressors (here, $D$), saving the residuals (here, the time-demeaned $y$ or $M_{[D]}y$, because regression on a constant just demeans the variables), then
2. regressing the $X$ on $D$ and saving the residuals $M_{[D]}X$, and
3. regressing the residuals onto each other, $M_{[D]}y$ on $M_{[D]}X$.

The second version is much more widely used, because typical panel data sets may have thousands of panel units $N$, so that the first approach would require you to run a regression with thousands of regressors, which is not a good idea numerically even nowadays with fast computers, as computing the inverse of $(D :X)'(D: X)$ would be very expensive, whereas time-demeaning $y$ and $X$ is of little cost.

Answer 2

Here is a simplified version of my first answer, which I believe is less practically relevant, but possibly easier to "sell" for classroom use.

The regressions $$y_i = \beta_1 + \sum_{j=2}^K\beta_jx_{ij} + \epsilon_i$$ and $$y_i-\bar{y} = \sum^K_{j=2}\beta_j(x_{ij} - \bar{x}_j) + \tilde{\epsilon}_i$$ yield identical $\widehat{\beta}_j$, $j=2,\ldots,K$. This can be seen as follows: take $\mathbf{x}_1=\mathbf{1}:=(1,\ldots,1)'$ and hence $$ M_\mathbf{1}=I-\mathbf{1}(\mathbf{1}'\mathbf{1})^{-1}\mathbf{1}'=I-\frac{\mathbf{1}\mathbf{1}'}{n}, $$ so that $$M_{\mathbf{1}}\mathbf{x}_j=\mathbf{x}_j-\mathbf{1} n^{-1}\mathbf{1}'\mathbf{x}_j=\mathbf{x}_j-\mathbf{1}\bar{x}_j=:\mathbf{x}_j-\bar{\mathbf{x}}_j. $$ Hence, the residuals of a regression of variables on a constant, $M_{\mathbf{1}}\mathbf{x}_j$, are just the demeaned variables (the same logic of course applies to $y_i$).

Answer 3

Here is another, more indirect, but I believe interesting one, namely the connection between different approaches to computing the partial autocorrelation coefficient of a stationary time series.

Definition 1

Consider the projection \begin{equation} \hat{Y}_{t}-\mu=\alpha^{(m)}_1(Y_{t-1}-\mu)+\alpha^{(m)}_2(Y_{t-2}-\mu)+\ldots+\alpha^{(m)}_m(Y_{t-m}-\mu) \end{equation} The $m$th partial autocorrelation equals $\alpha^{(m)}_m$.

It thus gives the influence of the $m$th lag on $Y_t$ \emph{after controlling for} $Y_{t-1},\ldots,Y_{t-m+1}$. Contrast this with $\rho_m$, that gives the `raw' correlation of $Y_t$ and $Y_{t-m}$.

How do we find the $\alpha^{(m)}_j$? Recall that a fundamental property of a regression of $Z_t$ on regressors $X_t$ is that the coefficients are such that regressors and residuals are uncorrelated. In a population regression this condition is then stated in terms of population correlations. Then: \begin{equation} E[X_t(Z_t-X_t^\top\mathbf{\alpha}^{(m)})]=0 \end{equation} Solving for $\mathbf{\alpha}^{(m)}$ we find the linear projection coefficients \begin{equation} \mathbf{\alpha}^{(m)}=[E(X_tX_t^\top)]^{-1}E[X_tZ_t] \end{equation} Applying this formula to $Z_t=Y_t-\mu$ and $$X_t=[(Y_{t-1}-\mu),(Y_{t-2}-\mu),\ldots,(Y_{t-m}-\mu)]^\top$$ we have $$ E(X_tX_t^\top)=\left(\begin{array}{cccc} \gamma_{0} & \gamma_{1}&\cdots& \gamma_{m-1}\\ \gamma_{1}& \gamma_{0} & \cdots &\gamma_{m-2}\\ \vdots & \vdots & \ddots &\vdots\\ \gamma_{m-1}&\gamma_{m-2} & \cdots & \gamma_{0}\\ \end{array} \right) $$ Also, $$ E(X_tZ_t)=\left( \begin{array}{c} \gamma_1 \\ \vdots \\ \gamma_m \\ \end{array} \right) $$ Hence, \begin{equation} \mathbf{\alpha}^{(m)}=\left(\begin{array}{cccc} \gamma_{0} & \gamma_{1}&\cdots& \gamma_{m-1}\\ \gamma_{1}& \gamma_{0} & \cdots &\gamma_{m-2}\\ \vdots & \vdots & \ddots &\vdots\\ \gamma_{m-1}&\gamma_{m-2} & \cdots & \gamma_{0}\\ \end{array} \right)^{-1}\left( \begin{array}{c} \gamma_1 \\ \vdots \\ \gamma_m \\ \end{array} \right)\end{equation} The $m$th partial correlation then is the last element of the vector $\mathbf{\alpha}^{(m)}$.

So, we sort of run a multiple regression and find one coefficient of interest while controlling for the others.

Definition 2

The $m$th partial correlation is the correlation of the prediction error of $Y_{t+m}$ predicted with $Y_{t-1},\ldots,Y_{t-m+1}$ with the prediction error of $Y_{t}$ predicted with $Y_{t-1},\ldots,Y_{t-m+1}$.

So, we sort of first control for the intermediate lags and then compute the correlation of the residuals.