How to show least square estimator is not BLUE when residual is dependent of each other?

Question

Suppose we have $y=X\beta+\varepsilon$, where $\varepsilon \sim (0,\sigma^2V)$, $\sigma^2$ unknown but $V$ known (we can assume a valid $V$ for this model).

Then, by general least square, we can find $Cov(\hat{\beta}_{GLS},\hat{\beta}_{GLS}^T)=\sigma^2(X^TV^{-1}X)^{-1}$ and $Cov(\hat{\beta}_{LS},\hat{\beta}_{LS}^T)=\sigma^2(X^TX)^{-1}X^TVX(X^TX)^{-1})$.

Then, how to show $Cov(\hat{\beta}_{LS},\hat{\beta}_{LS}^T)-Cov(\hat{\beta}_{GLS},\hat{\beta}_{GLS}^T)\succeq0$ (Positive Semi-Definite)? That is, how to prove $\hat{\beta}_{LS}$ is not BLUE in this case?

Answer 1

Assuming $X$ is non-random and $V$ is positive definite. Suppose $\Omega=\sigma^2V$.

Variance-covariance matrix of $\hat\beta_{GLS}=(X^T\Omega^{-1}X)^{-1}X^T\Omega^{-1}y$ is then

\begin{align} \operatorname{Var}(\hat\beta_{GLS})&=(X^T \Omega^{-1}X)^{-1}X^T\Omega^{-1}\Omega\,\Omega^{-1}X(X^T\Omega^{-1}X)^{-1} \tag{1} \\&=(X^T \Omega^{-1}X)^{-1} \end{align}

And that of $\hat\beta_{OLS}=(X^TX)^{-1}X^Ty$ is

$$\operatorname{Var}(\hat\beta_{OLS})=(X^TX)^{-1}X^T\Omega X(X^TX)^{-1} \tag{2}$$

Now verify using $(1)$ and $(2)$ that

$$\operatorname{Var}(\hat\beta_{OLS})-\operatorname{Var}(\hat\beta_{GLS})=A\Omega A^T\,,$$

where $$A=(X^TX)^{-1}X^T-(X^T\Omega^{-1}X)^{-1}X^T\Omega^{-1}$$

As $\Omega$ is positive definite, the matrix $A\Omega A^T$ is positive semi-definite.

This shows that $\hat\beta_{GLS}$ is better than $\hat\beta_{OLS}$.