# Proving efficiency of OLS over GLS

I'm trying to prove the efficiency of OLS over GLS when the covariance matrix of the error $$\varepsilon$$ is mistakenly assumed to be $$\sigma^2\Sigma$$ instead of $$\sigma^2 I$$. After deriving the variances, what I have so far is: $$Var(\beta^{ols}) = \sigma^2(X'X)^{-1}$$ and $$Var(\beta^{gls}) = \sigma^2(X'\Sigma^{-1} X)^{-1}$$.

Want to show $$Var(\beta^{gls}) - Var(\beta^{ols})$$ is psd. $$\implies \sigma^2(X'\Sigma^{-1} X)^{-1} - \sigma^2(X'X)^{-1} \geq 0\\ \implies (X'\Sigma^{-1} X)^{-1} - (X'X)^{-1} \geq 0 \\ \iff X'X - X'\Sigma^{-1} X \geq 0$$ But this is where I don't know how to proceed. First I tried $$X'(I-\Sigma^{-1})X \geq 0$$ but got stuck. Any hints on how to continue?

Update: After one of the comments, I want to double check the variance of $$\beta^{gls}$$: $$Var(\beta^{gls}) = Var[(X'\Sigma^{-1}X)^{-1} X'\Sigma^{-1}\varepsilon]\\ =(X'\Sigma^{-1}X)^{-1} X'\Sigma^{-1} Var(\varepsilon) \Sigma^{-1} X(X'\Sigma^{-1}X)^{-1}$$ Here's is where I think I made the mistake, I replaced the variance of $$\varepsilon$$ as the one that we think is "right" (i.e. $$\sigma^{2}\Sigma)$$. Should I replace it with the real one (i.e. $$\sigma^2I$$)?

• It seems to me the Gauss-Markov theorem implies this as part of its more general conclusion about the BLUE property of OLS, or am I missing something? Mar 30 '19 at 18:30
• @jbowman I Agree with you, I saw the proof of Gauss-Markov on Wikipedia, I'm just having a hard time figuring out what would the D matrix used in the proof specifically for GLS. Mar 30 '19 at 18:44
• Perhaps you could be able to use the properties of $\Sigma$? After all, it is not just any matrix, it must have some specific features. Mar 30 '19 at 20:48
• When the variance matrix of $\varepsilon$ is $\sigma^2 I$, then $Var(\beta^{gls}) \neq \sigma^2(X'\Sigma^{-1} X)^{-1}$. Mar 30 '19 at 20:59
• @RichardHardy Yes, it must be positive semidefinite and symmetric, which means it can be written $\Sigma = \Sigma^{\frac{1}{2}} \Sigma^{\frac{1}{2}}$, but I still cannot get it. Mar 30 '19 at 20:59

This is what I ended up doing: $$Var(\beta^{gls}) = Var[(X'\Sigma^{-1}X)^{-1} X'\Sigma^{-1}\varepsilon]\\ =(X'\Sigma^{-1}X)^{-1} X'\Sigma^{-1} Var(\varepsilon) \Sigma^{-1} X(X'\Sigma^{-1}X)^{-1}\\ = \sigma^2 (X'\Sigma^{-1}X)^{-1} X'\Sigma^{-1} \Sigma^{-1} X(X'\Sigma^{-1}X)^{-1}\\$$
Want to show $$Var(\beta^{gls}) - Var(\beta^{ols})$$ is psd. $$\implies \sigma^2 (X'\Sigma^{-1}X)^{-1} X'\Sigma^{-1} \Sigma^{-1} X(X'\Sigma^{-1}X)^{-1} - \sigma^2(X'X)^{-1} \geq 0\\ \iff X'X - X'\Sigma^{-1}X( X'\Sigma^{-1} \Sigma^{-1} X)^{-1}X'\Sigma^{-1}X \geq 0\\ \implies X'(I-\Sigma^{-1}X( X'\Sigma^{-1} \Sigma^{-1} X)^{-1}X'\Sigma^{-1})X \geq 0\\ \implies X'MX\geq0$$ Where $$M$$ is the residual maker matrix, and since $$M$$ is psd, the statement holds.
It is known that $$V(\beta^{gls}-\beta^{ols}|X)$$ is positive semidefinite. Here it turns out that $$V(\beta^{gls}-\beta^{ols}|X)=V(\beta^{gls}|X)-V(\beta^{ols}|X),$$ hence the conclusion that $$\beta^{ols}$$ is relatively more efficient than $$\beta^{gls}$$ if $$V(\varepsilon|X)=\sigma^2I_N$$.
• How do we know that $V(\beta^{gls} - \beta{ols})|X)$ is psd? Apr 2 '19 at 12:56
• Any variance matrix is positive semidefinite. This follows from its definition: $V[Z]=E[(Z−E[Z])(Z−E[Z])^T]$. Take $Z = \beta^{gls}-\beta^{ols}|X$ Apr 2 '19 at 18:05
• I agree with the variance statement, but then what would stop me from saying that $V(\beta^{OLS} - \beta^{GLS})$ is positive semidefinite too and apply the same argument? Anyhow, I think I got the proof and it was because of your initial comment, so I really appreciate your help. I'll post it as soon as I have time. Apr 3 '19 at 0:06
• Yes, indeed, in general $$V(\beta^{gls}-\beta^{ols}|X)=V(\beta^{ols}-\beta^{gls}|X)=V(\beta^{ols})-2cov(\beta^{gls},\beta^{ols}|X)+V(\beta^{ols}|X).$$In this particular case, however, this last expression boilds down to the above equality. Apr 3 '19 at 6:38