# How to show least square estimator is not BLUE when residual is dependent of each other?

Suppose we have $$y=X\beta+\varepsilon$$, where $$\varepsilon \sim (0,\sigma^2V)$$, $$\sigma^2$$ unknown but $$V$$ known (we can assume a valid $$V$$ for this model).

Then, by general least square, we can find $$Cov(\hat{\beta}_{GLS},\hat{\beta}_{GLS}^T)=\sigma^2(X^TV^{-1}X)^{-1}$$ and $$Cov(\hat{\beta}_{LS},\hat{\beta}_{LS}^T)=\sigma^2(X^TX)^{-1}X^TVX(X^TX)^{-1})$$.

Then, how to show $$Cov(\hat{\beta}_{LS},\hat{\beta}_{LS}^T)-Cov(\hat{\beta}_{GLS},\hat{\beta}_{GLS}^T)\succeq0$$ (Positive Semi-Definite)? That is, how to prove $$\hat{\beta}_{LS}$$ is not BLUE in this case?

• Mar 25 at 12:21

Assuming $$X$$ is non-random and $$V$$ is positive definite. Suppose $$\Omega=\sigma^2V$$.

Variance-covariance matrix of $$\hat\beta_{GLS}=(X^T\Omega^{-1}X)^{-1}X^T\Omega^{-1}y$$ is then

\begin{align} \operatorname{Var}(\hat\beta_{GLS})&=(X^T \Omega^{-1}X)^{-1}X^T\Omega^{-1}\Omega\,\Omega^{-1}X(X^T\Omega^{-1}X)^{-1} \tag{1} \\&=(X^T \Omega^{-1}X)^{-1} \end{align}

And that of $$\hat\beta_{OLS}=(X^TX)^{-1}X^Ty$$ is

$$\operatorname{Var}(\hat\beta_{OLS})=(X^TX)^{-1}X^T\Omega X(X^TX)^{-1} \tag{2}$$

Now verify using $$(1)$$ and $$(2)$$ that

$$\operatorname{Var}(\hat\beta_{OLS})-\operatorname{Var}(\hat\beta_{GLS})=A\Omega A^T\,,$$

where $$A=(X^TX)^{-1}X^T-(X^T\Omega^{-1}X)^{-1}X^T\Omega^{-1}$$

As $$\Omega$$ is positive definite, the matrix $$A\Omega A^T$$ is positive semi-definite.

This shows that $$\hat\beta_{GLS}$$ is better than $$\hat\beta_{OLS}$$.