1
$\begingroup$

I'm trying to prove the efficiency of OLS over GLS when the covariance matrix of the error $\varepsilon$ is mistakenly assumed to be $\sigma^2\Sigma$ instead of $\sigma^2 I$. After deriving the variances, what I have so far is: $Var(\beta^{ols}) = \sigma^2(X'X)^{-1}$ and $Var(\beta^{gls}) = \sigma^2(X'\Sigma^{-1} X)^{-1}$.

Want to show $Var(\beta^{gls}) - Var(\beta^{ols})$ is psd. $$\implies \sigma^2(X'\Sigma^{-1} X)^{-1} - \sigma^2(X'X)^{-1} \geq 0\\ \implies (X'\Sigma^{-1} X)^{-1} - (X'X)^{-1} \geq 0 \\ \iff X'X - X'\Sigma^{-1} X \geq 0 $$ But this is where I don't know how to proceed. First I tried $X'(I-\Sigma^{-1})X \geq 0$ but got stuck. Any hints on how to continue?

Update: After one of the comments, I want to double check the variance of $\beta^{gls}$: $$Var(\beta^{gls}) = Var[(X'\Sigma^{-1}X)^{-1} X'\Sigma^{-1}\varepsilon]\\ =(X'\Sigma^{-1}X)^{-1} X'\Sigma^{-1} Var(\varepsilon) \Sigma^{-1} X(X'\Sigma^{-1}X)^{-1} $$ Here's is where I think I made the mistake, I replaced the variance of $\varepsilon$ as the one that we think is "right" (i.e. $\sigma^{2}\Sigma) $. Should I replace it with the real one (i.e. $\sigma^2I$)?

Improve this question
$\endgroup$
7
  • $\begingroup$ It seems to me the Gauss-Markov theorem implies this as part of its more general conclusion about the BLUE property of OLS, or am I missing something? $\endgroup$
    – jbowman
    Commented Mar 30, 2019 at 18:30
  • $\begingroup$ @jbowman I Agree with you, I saw the proof of Gauss-Markov on Wikipedia, I'm just having a hard time figuring out what would the D matrix used in the proof specifically for GLS. $\endgroup$
    – user280809
    Commented Mar 30, 2019 at 18:44
  • $\begingroup$ Perhaps you could be able to use the properties of $\Sigma$? After all, it is not just any matrix, it must have some specific features. $\endgroup$
    – Richard Hardy
    Commented Mar 30, 2019 at 20:48
  • $\begingroup$ When the variance matrix of $\varepsilon$ is $\sigma^2 I$, then $Var(\beta^{gls}) \neq \sigma^2(X'\Sigma^{-1} X)^{-1}$. $\endgroup$
    – Bertrand
    Commented Mar 30, 2019 at 20:59
  • $\begingroup$ @RichardHardy Yes, it must be positive semidefinite and symmetric, which means it can be written $\Sigma = \Sigma^{\frac{1}{2}} \Sigma^{\frac{1}{2}}$, but I still cannot get it. $\endgroup$
    – user280809
    Commented Mar 30, 2019 at 20:59

2 Answers 2

Reset to default
1
$\begingroup$

This is what I ended up doing: $$Var(\beta^{gls}) = Var[(X'\Sigma^{-1}X)^{-1} X'\Sigma^{-1}\varepsilon]\\ =(X'\Sigma^{-1}X)^{-1} X'\Sigma^{-1} Var(\varepsilon) \Sigma^{-1} X(X'\Sigma^{-1}X)^{-1}\\ = \sigma^2 (X'\Sigma^{-1}X)^{-1} X'\Sigma^{-1} \Sigma^{-1} X(X'\Sigma^{-1}X)^{-1}\\ $$

Want to show $Var(\beta^{gls}) - Var(\beta^{ols})$ is psd. $$\implies \sigma^2 (X'\Sigma^{-1}X)^{-1} X'\Sigma^{-1} \Sigma^{-1} X(X'\Sigma^{-1}X)^{-1} - \sigma^2(X'X)^{-1} \geq 0\\ \iff X'X - X'\Sigma^{-1}X( X'\Sigma^{-1} \Sigma^{-1} X)^{-1}X'\Sigma^{-1}X \geq 0\\ \implies X'(I-\Sigma^{-1}X( X'\Sigma^{-1} \Sigma^{-1} X)^{-1}X'\Sigma^{-1})X \geq 0\\ \implies X'MX\geq0 $$ Where $M$ is the residual maker matrix, and since $M$ is psd, the statement holds.

Improve this answer
$\endgroup$
0
$\begingroup$

It is known that $V(\beta^{gls}-\beta^{ols}|X)$ is positive semidefinite. Here it turns out that $$V(\beta^{gls}-\beta^{ols}|X)=V(\beta^{gls}|X)-V(\beta^{ols}|X),$$ hence the conclusion that $\beta^{ols}$ is relatively more efficient than $\beta^{gls}$ if $V(\varepsilon|X)=\sigma^2I_N$.

Improve this answer
$\endgroup$
4
  • $\begingroup$ How do we know that $V(\beta^{gls} - \beta{ols})|X)$ is psd? $\endgroup$
    – user280809
    Commented Apr 2, 2019 at 12:56
  • $\begingroup$ Any variance matrix is positive semidefinite. This follows from its definition: $V[Z]=E[(Z−E[Z])(Z−E[Z])^T]$. Take $Z = \beta^{gls}-\beta^{ols}|X $ $\endgroup$
    – Bertrand
    Commented Apr 2, 2019 at 18:05
  • $\begingroup$ I agree with the variance statement, but then what would stop me from saying that $V(\beta^{OLS} - \beta^{GLS})$ is positive semidefinite too and apply the same argument? Anyhow, I think I got the proof and it was because of your initial comment, so I really appreciate your help. I'll post it as soon as I have time. $\endgroup$
    – user280809
    Commented Apr 3, 2019 at 0:06
  • $\begingroup$ Yes, indeed, in general $$V(\beta^{gls}-\beta^{ols}|X)=V(\beta^{ols}-\beta^{gls}|X)=V(\beta^{ols})-2cov(\beta^{gls},\beta^{ols}|X)+V(\beta^{ols}|X).$$In this particular case, however, this last expression boilds down to the above equality. $\endgroup$
    – Bertrand
    Commented Apr 3, 2019 at 6:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.