13
$\begingroup$

I am supposed to teach the Frish Waugh theorem in econometrics, which I have not studied.

I have understood the maths behind it and I hope the idea too "the coefficient you get for a particular coefficient from a multiple linear model is equal to the coefficient of the simple regression model if you "eliminate" the influence of the other regressors". So the theoretical idea is kind of cool. (If I totally misunderstood I do welcome a correction)

But does it have some classical/practical usages ?

EDIT : I have accepted an answer, but am still willing to have new ones that bring other examples/applications.

$\endgroup$
  • 4
    $\begingroup$ An obvious one would be added variable plots? $\endgroup$ – Silverfish Mar 16 '16 at 0:06
  • 1
    $\begingroup$ Dougherty's Introduction to Econometrics mentions another example of using the Frisch-Waugh-Lovell theorem. In the early days of econometric analysis of time series, it was quite common in models where variables had deterministic time trends to detrend them all before regressing. But by FWL, you get the same coefficients simply by including a time trend as a regressor, and moreover this gives the "correct" standard errors, since it acknowledges that 1 df has thereby been consumed. $\endgroup$ – Silverfish Mar 16 '16 at 0:15
  • 1
    $\begingroup$ Dougherty does warn against the procedure, so in that respect it's not a great example, even though it's an instructive one. Economic variables often seem to be difference-stationary rather than trend-stationary, so this kind of attempted detrending doesn't work and can result in spurious regressions. $\endgroup$ – Silverfish Mar 16 '16 at 0:17
  • 1
    $\begingroup$ @Silverfish: FWL is a purely algebraic technique, so the issue of whether extracting a deterministic trend is "right" given the underlying DGP is no doubt important, but imho unrelated to FWL, so in that sense your example is a perfectly valid one for OPs question about the two ways to obtain point estimates. $\endgroup$ – Christoph Hanck Mar 16 '16 at 13:50
  • 2
    $\begingroup$ I have exploited this relationship in many posts, primarily for conceptual purposes and to provide interesting examples of regression phenomena. See, inter alia, stats.stackexchange.com/a/46508, stats.stackexchange.com/a/113207, and stats.stackexchange.com/a/71257. $\endgroup$ – whuber Mar 16 '16 at 16:59
11
$\begingroup$

Consider the fixed effects panel data model, also known as Least Squares Dummy Variables (LSDV) model.

$b_{LSDV}$ can be computed by directly applying OLS to the model $$y=X\beta+D\alpha+\epsilon,$$ where $D$ is a $NT\times N$ matrix of dummies and $\alpha$ represent the individual-specific fixed effects.

Another way to compute $b_{LSDV}$ is to apply the so called within transformation to the usual model in order to obtain a demeaned version of it, i.e. $$M_{[D]}y=M_{[D]}X\beta+M_{[D]}\epsilon.$$ Here, $M_{[D]}=I-D(D'D)^{-1}D'$, the residual maker matrix of a regression on $D$.

By the Frisch-Waugh-Lovell theorem, the two are equivalent, as FWL says that you can compute a subset of regression coefficients of a regression (here, $\hat\beta$) by

  1. regressing $y$ on the other regressors (here, $D$), saving the residuals (here, the time-demeaned $y$ or $M_{[D]}y$, because regression on a constant just demeans the variables), then
  2. regressing the $X$ on $D$ and saving the residuals $M_{[D]}X$, and
  3. regressing the residuals onto each other, $M_{[D]}y$ on $M_{[D]}X$.

The second version is much more widely used, because typical panel data sets may have thousands of panel units $N$, so that the first approach would require you to run a regression with thousands of regressors, which is not a good idea numerically even nowadays with fast computers, as computing the inverse of $(D :X)'(D: X)$ would be very expensive, whereas time-demeaning $y$ and $X$ is of little cost.

$\endgroup$
  • $\begingroup$ Thanks a lot, this is the kind of answer I was looking for, even though it is a bit advanced for me to actually use it. So your answer is fine with me, but I would be happy if I have other ones, am I supposed to accept yours ? $\endgroup$ – Anthony Martin Mar 16 '16 at 16:15
  • $\begingroup$ If it helped it would be appropriate to do so. But accepting will reduce your chances of getting better answers, so you may consider to wait before accepting this one. A bounty would further increase your chances of getting more answers - given that there are not enough users on CV who regularly answer questions given the amount of questions, even a single answer may lead other active users to conclude that the questions has been dealt with. (I did post a somewhat simpler answer below.) $\endgroup$ – Christoph Hanck Mar 16 '16 at 16:31
4
$\begingroup$

Here is a simplified version of my first answer, which I believe is less practically relevant, but possibly easier to "sell" for classroom use.

The regressions $$y_i = \beta_1 + \sum_{j=2}^K\beta_jx_{ij} + \epsilon_i$$ and $$y_i-\bar{y} = \sum^K_{j=2}\beta_j(x_{ij} - \bar{x}_j) + \tilde{\epsilon}_i$$ yield identical $\widehat{\beta}_j$, $j=2,\ldots,K$. This can be seen as follows: take $\mathbf{x}_1=\mathbf{1}:=(1,\ldots,1)'$ and hence $$ M_\mathbf{1}=I-\mathbf{1}(\mathbf{1}'\mathbf{1})^{-1}\mathbf{1}'=I-\frac{\mathbf{1}\mathbf{1}'}{n}, $$ so that $$M_{\mathbf{1}}\mathbf{x}_j=\mathbf{x}_j-\mathbf{1} n^{-1}\mathbf{1}'\mathbf{x}_j=\mathbf{x}_j-\mathbf{1}\bar{x}_j=:\mathbf{x}_j-\bar{\mathbf{x}}_j. $$ Hence, the residuals of a regression of variables on a constant, $M_{\mathbf{1}}\mathbf{x}_j$, are just the demeaned variables (the same logic of course applies to $y_i$).

$\endgroup$
4
$\begingroup$

Here is another, more indirect, but I believe interesting one, namely the connection between different approaches to computing the partial autocorrelation coefficient of a stationary time series.

Definition 1

Consider the projection \begin{equation} \hat{Y}_{t}-\mu=\alpha^{(m)}_1(Y_{t-1}-\mu)+\alpha^{(m)}_2(Y_{t-2}-\mu)+\ldots+\alpha^{(m)}_m(Y_{t-m}-\mu) \end{equation} The $m$th partial autocorrelation equals $\alpha^{(m)}_m$.

It thus gives the influence of the $m$th lag on $Y_t$ \emph{after controlling for} $Y_{t-1},\ldots,Y_{t-m+1}$. Contrast this with $\rho_m$, that gives the `raw' correlation of $Y_t$ and $Y_{t-m}$.

How do we find the $\alpha^{(m)}_j$? Recall that a fundamental property of a regression of $Z_t$ on regressors $X_t$ is that the coefficients are such that regressors and residuals are uncorrelated. In a population regression this condition is then stated in terms of population correlations. Then: \begin{equation} E[X_t(Z_t-X_t^\top\mathbf{\alpha}^{(m)})]=0 \end{equation} Solving for $\mathbf{\alpha}^{(m)}$ we find the linear projection coefficients \begin{equation} \mathbf{\alpha}^{(m)}=[E(X_tX_t^\top)]^{-1}E[X_tZ_t] \end{equation} Applying this formula to $Z_t=Y_t-\mu$ and $$X_t=[(Y_{t-1}-\mu),(Y_{t-2}-\mu),\ldots,(Y_{t-m}-\mu)]^\top$$ we have $$ E(X_tX_t^\top)=\left(\begin{array}{cccc} \gamma_{0} & \gamma_{1}&\cdots& \gamma_{m-1}\\ \gamma_{1}& \gamma_{0} & \cdots &\gamma_{m-2}\\ \vdots & \vdots & \ddots &\vdots\\ \gamma_{m-1}&\gamma_{m-2} & \cdots & \gamma_{0}\\ \end{array} \right) $$ Also, $$ E(X_tZ_t)=\left( \begin{array}{c} \gamma_1 \\ \vdots \\ \gamma_m \\ \end{array} \right) $$ Hence, \begin{equation} \mathbf{\alpha}^{(m)}=\left(\begin{array}{cccc} \gamma_{0} & \gamma_{1}&\cdots& \gamma_{m-1}\\ \gamma_{1}& \gamma_{0} & \cdots &\gamma_{m-2}\\ \vdots & \vdots & \ddots &\vdots\\ \gamma_{m-1}&\gamma_{m-2} & \cdots & \gamma_{0}\\ \end{array} \right)^{-1}\left( \begin{array}{c} \gamma_1 \\ \vdots \\ \gamma_m \\ \end{array} \right)\end{equation} The $m$th partial correlation then is the last element of the vector $\mathbf{\alpha}^{(m)}$.

So, we sort of run a multiple regression and find one coefficient of interest while controlling for the others.

Definition 2

The $m$th partial correlation is the correlation of the prediction error of $Y_{t+m}$ predicted with $Y_{t-1},\ldots,Y_{t-m+1}$ with the prediction error of $Y_{t}$ predicted with $Y_{t-1},\ldots,Y_{t-m+1}$.

So, we sort of first control for the intermediate lags and then compute the correlation of the residuals.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.