How to prove positive (semi-)definitness in matrix notation without numbers I'd like to show that $$V[\hat\beta_{OLS}]-V[\hat\beta_{GLS}]=\sigma^2(X'X)^{-1}(X'\Omega X)(X'X)^{-1}-\sigma^2(X'\Omega X)^{-1}\geq 0$$ is positive (semi-) definite. $\Omega$ and $X$ are square-matrices. I know how to check for positive (semi-) definiteness with eigenvalues if the matrices contain numbers, but I wonder if there is a quick way to check if the above expression is positive (semi-) definite when we only have matrix notation without numbers?
 A: Note that $X$ is generally not square, unless you have as many regressors as observations. In that case - see the derivation at the end of my answer - OLS and GLS are equally efficient.
Notice also a mistake in the above expression in that the correct variance of GLS is
$$
\sigma^2(X'\Omega^{-1} X)^{-1}
$$
A standard result in matrix algebra tells out that
$$
A\geq B\Leftrightarrow B^{-1}-A^{-1}\geq0$$
(much like $3>2$ and $1/2>1/3$). Hence, we may also establish that ($\sigma^2$ is just a scale factor which cancels out of the comparison)
$$
X'\Omega^{-1} X-X'X(X'\Omega X)^{-1}X'X\geq 0
$$
Rearrange the lhs to
$$
X'(\Omega^{-1}-X(X'\Omega X)^{-1}X')X
$$
or, with $\Omega^{1/2}$ a symmetric matrix square root of $\Omega$ and $\Omega^{-1/2}$ its inverse,
$$
X'(\Omega^{-1/2}\Omega^{-1/2}-\Omega^{-1/2}\Omega^{1/2}X(X'\Omega X)^{-1}X'\Omega^{1/2}\Omega^{-1/2})X
$$
or
$$
X'\Omega^{-1/2}(I-\Omega^{1/2}X(X'\Omega X)^{-1}X'\Omega^{1/2})\Omega^{-1/2}X
$$
or
$$
X'\Omega^{-1/2}(I-\Omega^{1/2}X(X'\Omega^{1/2}\Omega^{1/2} X)^{-1}X'\Omega^{1/2})\Omega^{-1/2}X
$$
Now let $C:=\Omega^{-1/2}X$ and $D:=\Omega^{1/2}X$ so that we may write
$$
C'(I-D(D'D)^{-1}D')C
$$
Now, $M_D:=I-D(D'D)^{-1}D'$ is a residual-maker, hence, symmetric and idempotent projection matrix. Let $c:=Ce$ for some nonzero vector $e$. Then, for $f:=M_Dc$,
$$
c'M_Dc=c'M_D'M_Dc\equiv f'f\geq0,
$$
as $f'f=\sum_if_i^2$ is a sum of squares.
Now, if $X$ actually were square (and invertible), we would have
$$
X'X(X'\Omega X)^{-1}X'X=X'XX^{-1}\Omega^{-1}(X')^{-1}X'X=X'\Omega^{-1} X
$$
so that OLS and GLS would be equally efficient!
In fact, they would be numerically identical,
as
$$
(X'\Omega^{-1} X)^{-1}X'\Omega^{-1}y=X^{-1}\Omega X'^{-1}X'\Omega^{-1}y=X^{-1}y=X^{-1}X'^{-1}X'y=(X'X)^{-1}X'y
$$
