I have come across this question in a textbook: I have a linear model $Y=Xb+u$ with for instance autocorrelation, in order to introduce GLS $Y^*=X^*b+u^*$ (with $Z^* = \Omega^{-1/2}Z$).

Then an additional question is to show that

  • $\hat{b}_{GLS} = (X^T\Omega^{-1}X)^{-1} X^T\Omega^{-1}Y$, and
  • $\hat{\sigma}^2_{GLS} = \dfrac{\hat{u}^{*T}\hat{u}^*}{N-(k+1)}$

are independent.

And I have no precise idea to do it. I can show that their covariance is equal to zero, but unless I assume Normal distributions this does not help me.

Am I supposed to aim at joint density or is there something better?

  • 1
    $\begingroup$ Someone edited my question with the "unless I assume" although it was "as these are non". I feel that even if I assume normality (that is of the residuals), these estimates are still not normal and therefore, I cannot conclude with just a zero covariance. I therefore think this edit is misleading (and you probably see I am not sure the best place to signal it) $\endgroup$ – Anthony Martin Apr 14 '16 at 12:46
  • $\begingroup$ Typically, the predictors are assumed fixed, or conditioned on, so assuming the error vector to be normal does imply the estimators are normal. Are you considering X fixed? $\endgroup$ – ekvall Apr 14 '16 at 13:28
  • $\begingroup$ Yes i am, but $\hat{\sigma}^2_{GLS}$ then follows a $\chi^2$ distribution no ? $\endgroup$ – Anthony Martin Apr 14 '16 at 13:32
  • $\begingroup$ I gave an answer that I hope may help. I think you should also tag this question as self-study. $\endgroup$ – ekvall Apr 14 '16 at 14:11

Edit It's been a while and the OP has provided an answer, so here is a more detailed answer than was originally provided due to the homework-nature of the problem. I use a somewhat simplified notation where $Y = X\beta +u, u\sim N(0, \Omega)$.

We first notice that since $\hat{\sigma}^2$ is a function of $\hat{u}$, it suffices to prove that $\hat{u}$ is independent of $\hat{\beta}$.

Let $P = XH$ and $Q = I - P$, where $H = (X'\Omega^{-1}X)^{-1}X'\Omega^{-1}$. Then $\hat{\beta}=HY$, $\hat{\mu} = PY$ and $\hat{u}=QY$.

Since $Y \sim N(X\beta, \Omega)$, we have $\hat{\beta} \sim N(\beta, H\Omega H')$ and $\hat{u}\sim N(0, Q\Omega Q')$. Thus, it suffices to prove that $\operatorname{Cov}(\hat{\beta}, \hat{u})=0$. Moreover, assuming $X$ is full column rank, we have $$\operatorname{Cov}(\hat{\mu}, \hat{u}) = \operatorname{Cov}(X\hat{\beta}, \hat{u}) = X\operatorname{Cov}(\hat{\beta}, \hat{u})=0 \iff \operatorname{Cov}(\hat{\beta}, \hat{u}) = 0$$


\begin{align}\operatorname{Cov}(\hat{\mu}, \hat{u}) &= \operatorname{Cov}(PY, QY) \\ &= P\Omega Q' \\ &= P\Omega - P\Omega P' \\ &= X(X'\Omega^{-1}X)^{-1}X' - X(X'\Omega^{-1}X)^{-1}X'\Omega X(X'\Omega^{-1}X)^{-1}X' \\ &= X(X'\Omega^{-1}X)^{-1}X' - X(X'\Omega^{-1}X)^{-1}X'\\ &= 0, \end{align} which by the above iff condition is equivalent to what we wanted to show.


Building on the 6th point nicely given by Student001, here is what I have come to :

$cov(\widehat{b}_{MCG},\widehat{u}^*) = E\left[\left(\widehat{u}^*-E[\widehat{u}^*]\right) \left(\widehat{b}_{MCG}-E[\widehat{b}_{MCG}]\right)^T\right] \\ = E\left[\widehat{u}^*\left(\widehat{b}_{MCG}-b\right)^T\right] \\ = E\left[\widehat{u}^* \widehat{b}^{\,T}_{MCG}\right] - E[\widehat{u}^*] b^T \\ =E\left[M^*u^*\left((X^{*T}X^*)^{-1}X^{*T}u^*\right)^{T}\right] \\ =E\left[M^*u^*u^{*^T}X^*\left(X^{*^T}X^*\right)^{-1}\right] \\ =M^*E\left[u^*u^{*^T}\right] X^*(X^{*^T}X^*)^{-1}or\:E[u^*u^{*^T}]=\sigma_u^2 I_T \\ =\sigma^2_u M^*X^*(X^{*^T}X^*)^{-1} \\ %& &\text{ avec } M^*=I_T-X^*(X^{*T}X^*)^{-1}X^{*T} \\ = \sigma^2_u [X^*(X^{*^T}X^*)^{-1} -X^*(X^{*^T}X^*)^{-1}X^{*^T}X^*(X^{*^T}X^*)^{-1}] \\ =\sigma_u^2[X^*(X^{*^T}X^*)^{-1} -X^*(X^{*^T}X^*)^{-1}] \\ =0$

Then, as with the right assumption we have that $ \hat{u}^*$ is normal, and $\widehat{b}_{MCG}$ is too, they are independent.

An last, as $\hat{\sigma}_{GLS}^2$ is a function of $ \hat{u}^*$, $\widehat{b}_{MCG}$ and $\hat{\sigma}_{GLS}^2$ are independent.

By the way, is that of any practical help ?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.