2
$\begingroup$

The frisch-waugh-lovell theorem uses the following line in the proof:

$\hat\beta_2 = (X_2' M_{1}X_2)^{-1}X_2'M_{1}y$

How do we know

$(X_2' M_{1}X_2)^{-1}$

is invertible?

$\endgroup$
4
  • 1
    $\begingroup$ en.wikipedia.org/wiki/Positive-definite_matrix $\endgroup$ Mar 24, 2017 at 15:23
  • 1
    $\begingroup$ Do you know anything specific about $M_1$? Because that isn't invertible in general. $\endgroup$
    – Sycorax
    Mar 24, 2017 at 15:36
  • 2
    $\begingroup$ You don't know it's invertible--but you don't invert it in the first place. That's just a notation for solving a set of linear equations. Equivalently, think of it as a generalized inverse. $\endgroup$
    – whuber
    Mar 24, 2017 at 15:43
  • $\begingroup$ @whuber I would really appreciate it if you could have a look at my attempt at a proof below. Thank you. $\endgroup$ Mar 24, 2017 at 17:15

1 Answer 1

1
$\begingroup$

I think that we can show that the matrix is invertible if the full regressor matrix has full column rank, but please check my proof.

We are looking at a regression with $k_1+k_2$ regressors (counting a possible constant term) having a sample of size $n\geq k_1+k_2$. We asuume that the full regressor matrix $X = [X_1:X_2]$ has full-column rank equal to $k_1+k_2$. This means that $X_1$ has column rank $k_1$ and $X_2$ has column rank $k_2$.

Let $M_1 = I_{n} - P_1,\,\, P_1 = X_1(X_1'X_1)^{-1}X_1'$, the residual maker and orthogonal projection matrices respectively, symmetric and idempotent both. We want to show that the $n \times k_2$ matrix $M_1X_2$ has full column rank $k_2$, so that its Grammian matrix $[(M_1X_2)'(M_1X_2)] = X_2'M_1X_2$ is invertible.

Assume that $M_1X_2 = W$ has not full column rank. This means that at least two columns of $W=M_1X_2$ are linearly dependent, say columns $w_1 = x_{21}- P_1x_{21}$ and $w_2= x_{22} - P_1x_{22}$.
If they are linearly dependent it means that for some real scalar $a$ we have

$$w_1 = aw_2 \implies x_{21}- P_1x_{21}= ax_{22} - P_1ax_{22}$$

$$\implies x_{21}- ax_{22} = P_1(x_{21} - ax_{22})$$

So if $w_1$ and $w_2$ are linearly dependent, we have obtained that the Projection matrix $P_1$ must act as the identity matrix on the column vector $x_{21}- ax_{22}$. But in order to do so, this column vector must be inlcuded in the regressors used to form $P_1$.

But this implies that in the full regressor matrix $X$, there exists a column which is a linear combination of two other columns, and so $X$ has not full column rank, and so $X'X$ is not invertible to begin with.

So what I think I have proved is the following:

If the full regressor matrix $X$ has full column rank, then the matrix $M_1X_2$ as defined above has also full column rank.

$\endgroup$
1
  • 1
    $\begingroup$ +1 I believe you, because this theorem is an immediate consequence of an easily proven generalization to arbitrary partitions of the set of explanatory variables. (I describe this generalization geometrically at stats.stackexchange.com/a/113207/919 and algebraically, in terms that look similar to yours, at stats.stackexchange.com/a/46508/919.) $\endgroup$
    – whuber
    Mar 24, 2017 at 18:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.