1
$\begingroup$

I'm trying to prove the efficiency of OLS over GLS when the covariance matrix of the error $\varepsilon$ is mistakenly assumed to be $\sigma^2\Sigma$ instead of $\sigma^2 I$. After deriving the variances, what I have so far is: $Var(\beta^{ols}) = \sigma^2(X'X)^{-1}$ and $Var(\beta^{gls}) = \sigma^2(X'\Sigma^{-1} X)^{-1}$.

Want to show $Var(\beta^{gls}) - Var(\beta^{ols})$ is psd. $$\implies \sigma^2(X'\Sigma^{-1} X)^{-1} - \sigma^2(X'X)^{-1} \geq 0\\ \implies (X'\Sigma^{-1} X)^{-1} - (X'X)^{-1} \geq 0 \\ \iff X'X - X'\Sigma^{-1} X \geq 0 $$ But this is where I don't know how to proceed. First I tried $X'(I-\Sigma^{-1})X \geq 0$ but got stuck. Any hints on how to continue?

Update: After one of the comments, I want to double check the variance of $\beta^{gls}$: $$Var(\beta^{gls}) = Var[(X'\Sigma^{-1}X)^{-1} X'\Sigma^{-1}\varepsilon]\\ =(X'\Sigma^{-1}X)^{-1} X'\Sigma^{-1} Var(\varepsilon) \Sigma^{-1} X(X'\Sigma^{-1}X)^{-1} $$ Here's is where I think I made the mistake, I replaced the variance of $\varepsilon$ as the one that we think is "right" (i.e. $\sigma^{2}\Sigma) $. Should I replace it with the real one (i.e. $\sigma^2I$)?

$\endgroup$
7
  • $\begingroup$ It seems to me the Gauss-Markov theorem implies this as part of its more general conclusion about the BLUE property of OLS, or am I missing something? $\endgroup$
    – jbowman
    Mar 30, 2019 at 18:30
  • $\begingroup$ @jbowman I Agree with you, I saw the proof of Gauss-Markov on Wikipedia, I'm just having a hard time figuring out what would the D matrix used in the proof specifically for GLS. $\endgroup$
    – user280809
    Mar 30, 2019 at 18:44
  • $\begingroup$ Perhaps you could be able to use the properties of $\Sigma$? After all, it is not just any matrix, it must have some specific features. $\endgroup$ Mar 30, 2019 at 20:48
  • $\begingroup$ When the variance matrix of $\varepsilon$ is $\sigma^2 I$, then $Var(\beta^{gls}) \neq \sigma^2(X'\Sigma^{-1} X)^{-1}$. $\endgroup$
    – Bertrand
    Mar 30, 2019 at 20:59
  • $\begingroup$ @RichardHardy Yes, it must be positive semidefinite and symmetric, which means it can be written $\Sigma = \Sigma^{\frac{1}{2}} \Sigma^{\frac{1}{2}}$, but I still cannot get it. $\endgroup$
    – user280809
    Mar 30, 2019 at 20:59

2 Answers 2

1
$\begingroup$

This is what I ended up doing: $$Var(\beta^{gls}) = Var[(X'\Sigma^{-1}X)^{-1} X'\Sigma^{-1}\varepsilon]\\ =(X'\Sigma^{-1}X)^{-1} X'\Sigma^{-1} Var(\varepsilon) \Sigma^{-1} X(X'\Sigma^{-1}X)^{-1}\\ = \sigma^2 (X'\Sigma^{-1}X)^{-1} X'\Sigma^{-1} \Sigma^{-1} X(X'\Sigma^{-1}X)^{-1}\\ $$

Want to show $Var(\beta^{gls}) - Var(\beta^{ols})$ is psd. $$\implies \sigma^2 (X'\Sigma^{-1}X)^{-1} X'\Sigma^{-1} \Sigma^{-1} X(X'\Sigma^{-1}X)^{-1} - \sigma^2(X'X)^{-1} \geq 0\\ \iff X'X - X'\Sigma^{-1}X( X'\Sigma^{-1} \Sigma^{-1} X)^{-1}X'\Sigma^{-1}X \geq 0\\ \implies X'(I-\Sigma^{-1}X( X'\Sigma^{-1} \Sigma^{-1} X)^{-1}X'\Sigma^{-1})X \geq 0\\ \implies X'MX\geq0 $$ Where $M$ is the residual maker matrix, and since $M$ is psd, the statement holds.

$\endgroup$
0
$\begingroup$

It is known that $V(\beta^{gls}-\beta^{ols}|X)$ is positive semidefinite. Here it turns out that $$V(\beta^{gls}-\beta^{ols}|X)=V(\beta^{gls}|X)-V(\beta^{ols}|X),$$ hence the conclusion that $\beta^{ols}$ is relatively more efficient than $\beta^{gls}$ if $V(\varepsilon|X)=\sigma^2I_N$.

$\endgroup$
4
  • $\begingroup$ How do we know that $V(\beta^{gls} - \beta{ols})|X)$ is psd? $\endgroup$
    – user280809
    Apr 2, 2019 at 12:56
  • $\begingroup$ Any variance matrix is positive semidefinite. This follows from its definition: $V[Z]=E[(Z−E[Z])(Z−E[Z])^T]$. Take $Z = \beta^{gls}-\beta^{ols}|X $ $\endgroup$
    – Bertrand
    Apr 2, 2019 at 18:05
  • $\begingroup$ I agree with the variance statement, but then what would stop me from saying that $V(\beta^{OLS} - \beta^{GLS})$ is positive semidefinite too and apply the same argument? Anyhow, I think I got the proof and it was because of your initial comment, so I really appreciate your help. I'll post it as soon as I have time. $\endgroup$
    – user280809
    Apr 3, 2019 at 0:06
  • $\begingroup$ Yes, indeed, in general $$V(\beta^{gls}-\beta^{ols}|X)=V(\beta^{ols}-\beta^{gls}|X)=V(\beta^{ols})-2cov(\beta^{gls},\beta^{ols}|X)+V(\beta^{ols}|X).$$In this particular case, however, this last expression boilds down to the above equality. $\endgroup$
    – Bertrand
    Apr 3, 2019 at 6:38

This site is temporarily in read-only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .