I'd like to show that $$V[\hat\beta_{OLS}]-V[\hat\beta_{GLS}]=\sigma^2(X'X)^{-1}(X'\Omega X)(X'X)^{-1}-\sigma^2(X'\Omega X)^{-1}\geq 0$$ is positive (semi-) definite. $\Omega$ and $X$ are square-matrices. I know how to check for positive (semi-) definiteness with eigenvalues if the matrices contain numbers, but I wonder if there is a quick way to check if the above expression is positive (semi-) definite when we only have matrix notation without numbers?


1 Answer 1


Note that $X$ is generally not square, unless you have as many regressors as observations. In that case - see the derivation at the end of my answer - OLS and GLS are equally efficient.

Notice also a mistake in the above expression in that the correct variance of GLS is $$ \sigma^2(X'\Omega^{-1} X)^{-1} $$

A standard result in matrix algebra tells out that $$ A\geq B\Leftrightarrow B^{-1}-A^{-1}\geq0$$ (much like $3>2$ and $1/2>1/3$). Hence, we may also establish that ($\sigma^2$ is just a scale factor which cancels out of the comparison) $$ X'\Omega^{-1} X-X'X(X'\Omega X)^{-1}X'X\geq 0 $$ Rearrange the lhs to $$ X'(\Omega^{-1}-X(X'\Omega X)^{-1}X')X $$ or, with $\Omega^{1/2}$ a symmetric matrix square root of $\Omega$ and $\Omega^{-1/2}$ its inverse, $$ X'(\Omega^{-1/2}\Omega^{-1/2}-\Omega^{-1/2}\Omega^{1/2}X(X'\Omega X)^{-1}X'\Omega^{1/2}\Omega^{-1/2})X $$ or $$ X'\Omega^{-1/2}(I-\Omega^{1/2}X(X'\Omega X)^{-1}X'\Omega^{1/2})\Omega^{-1/2}X $$ or $$ X'\Omega^{-1/2}(I-\Omega^{1/2}X(X'\Omega^{1/2}\Omega^{1/2} X)^{-1}X'\Omega^{1/2})\Omega^{-1/2}X $$ Now let $C:=\Omega^{-1/2}X$ and $D:=\Omega^{1/2}X$ so that we may write $$ C'(I-D(D'D)^{-1}D')C $$ Now, $M_D:=I-D(D'D)^{-1}D'$ is a residual-maker, hence, symmetric and idempotent projection matrix. Let $c:=Ce$ for some nonzero vector $e$. Then, for $f:=M_Dc$, $$ c'M_Dc=c'M_D'M_Dc\equiv f'f\geq0, $$ as $f'f=\sum_if_i^2$ is a sum of squares.

Now, if $X$ actually were square (and invertible), we would have $$ X'X(X'\Omega X)^{-1}X'X=X'XX^{-1}\Omega^{-1}(X')^{-1}X'X=X'\Omega^{-1} X $$ so that OLS and GLS would be equally efficient!

In fact, they would be numerically identical, as $$ (X'\Omega^{-1} X)^{-1}X'\Omega^{-1}y=X^{-1}\Omega X'^{-1}X'\Omega^{-1}y=X^{-1}y=X^{-1}X'^{-1}X'y=(X'X)^{-1}X'y $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.