# Proving that the estimators of coefficients and variance in GLS model are independent

I have come across this question in a textbook: I have a linear model $Y=Xb+u$ with for instance autocorrelation, in order to introduce GLS $Y^*=X^*b+u^*$ (with $Z^* = \Omega^{-1/2}Z$).

Then an additional question is to show that

• $\hat{b}_{GLS} = (X^T\Omega^{-1}X)^{-1} X^T\Omega^{-1}Y$, and
• $\hat{\sigma}^2_{GLS} = \dfrac{\hat{u}^{*T}\hat{u}^*}{N-(k+1)}$

are independent.

And I have no precise idea to do it. I can show that their covariance is equal to zero, but unless I assume Normal distributions this does not help me.

Am I supposed to aim at joint density or is there something better?

• Someone edited my question with the "unless I assume" although it was "as these are non". I feel that even if I assume normality (that is of the residuals), these estimates are still not normal and therefore, I cannot conclude with just a zero covariance. I therefore think this edit is misleading (and you probably see I am not sure the best place to signal it) – Anthony Martin Apr 14 '16 at 12:46
• Typically, the predictors are assumed fixed, or conditioned on, so assuming the error vector to be normal does imply the estimators are normal. Are you considering X fixed? – ekvall Apr 14 '16 at 13:28
• Yes i am, but $\hat{\sigma}^2_{GLS}$ then follows a $\chi^2$ distribution no ? – Anthony Martin Apr 14 '16 at 13:32
• I gave an answer that I hope may help. I think you should also tag this question as self-study. – ekvall Apr 14 '16 at 14:11

Edit It's been a while and the OP has provided an answer, so here is a more detailed answer than was originally provided due to the homework-nature of the problem. I use a somewhat simplified notation where $Y = X\beta +u, u\sim N(0, \Omega)$.

We first notice that since $\hat{\sigma}^2$ is a function of $\hat{u}$, it suffices to prove that $\hat{u}$ is independent of $\hat{\beta}$.

Let $P = XH$ and $Q = I - P$, where $H = (X'\Omega^{-1}X)^{-1}X'\Omega^{-1}$. Then $\hat{\beta}=HY$, $\hat{\mu} = PY$ and $\hat{u}=QY$.

Since $Y \sim N(X\beta, \Omega)$, we have $\hat{\beta} \sim N(\beta, H\Omega H')$ and $\hat{u}\sim N(0, Q\Omega Q')$. Thus, it suffices to prove that $\operatorname{Cov}(\hat{\beta}, \hat{u})=0$. Moreover, assuming $X$ is full column rank, we have $$\operatorname{Cov}(\hat{\mu}, \hat{u}) = \operatorname{Cov}(X\hat{\beta}, \hat{u}) = X\operatorname{Cov}(\hat{\beta}, \hat{u})=0 \iff \operatorname{Cov}(\hat{\beta}, \hat{u}) = 0$$

But

\begin{align}\operatorname{Cov}(\hat{\mu}, \hat{u}) &= \operatorname{Cov}(PY, QY) \\ &= P\Omega Q' \\ &= P\Omega - P\Omega P' \\ &= X(X'\Omega^{-1}X)^{-1}X' - X(X'\Omega^{-1}X)^{-1}X'\Omega X(X'\Omega^{-1}X)^{-1}X' \\ &= X(X'\Omega^{-1}X)^{-1}X' - X(X'\Omega^{-1}X)^{-1}X'\\ &= 0, \end{align} which by the above iff condition is equivalent to what we wanted to show.

Building on the 6th point nicely given by Student001, here is what I have come to :

$cov(\widehat{b}_{MCG},\widehat{u}^*) = E\left[\left(\widehat{u}^*-E[\widehat{u}^*]\right) \left(\widehat{b}_{MCG}-E[\widehat{b}_{MCG}]\right)^T\right] \\ = E\left[\widehat{u}^*\left(\widehat{b}_{MCG}-b\right)^T\right] \\ = E\left[\widehat{u}^* \widehat{b}^{\,T}_{MCG}\right] - E[\widehat{u}^*] b^T \\ =E\left[M^*u^*\left((X^{*T}X^*)^{-1}X^{*T}u^*\right)^{T}\right] \\ =E\left[M^*u^*u^{*^T}X^*\left(X^{*^T}X^*\right)^{-1}\right] \\ =M^*E\left[u^*u^{*^T}\right] X^*(X^{*^T}X^*)^{-1}or\:E[u^*u^{*^T}]=\sigma_u^2 I_T \\ =\sigma^2_u M^*X^*(X^{*^T}X^*)^{-1} \\ %& &\text{ avec } M^*=I_T-X^*(X^{*T}X^*)^{-1}X^{*T} \\ = \sigma^2_u [X^*(X^{*^T}X^*)^{-1} -X^*(X^{*^T}X^*)^{-1}X^{*^T}X^*(X^{*^T}X^*)^{-1}] \\ =\sigma_u^2[X^*(X^{*^T}X^*)^{-1} -X^*(X^{*^T}X^*)^{-1}] \\ =0$

Then, as with the right assumption we have that $\hat{u}^*$ is normal, and $\widehat{b}_{MCG}$ is too, they are independent.

An last, as $\hat{\sigma}_{GLS}^2$ is a function of $\hat{u}^*$, $\widehat{b}_{MCG}$ and $\hat{\sigma}_{GLS}^2$ are independent.

By the way, is that of any practical help ?