0
$\begingroup$

I have been trying to derive if there is any relationship between the sum of squared residuals (SSE) from a model estimated with GLS, and the same model estimated with OLS. Professor Chung-Ming Kuan, on his lecture notes, states that the first is always the equal or greater than the second (bottom of page 80). This seems to come from a projection standpoint, but I am not able to grasp it, and cannot find this statement in any econometrics textbook. Is this relationship true, and may someone explain the reasoning behind it?

$\endgroup$
1
$\begingroup$

The least squares estimator $\hat{\beta}_T$ is defined to be the minimizer of the sum of squared residuals; this is expressed as: $$\|y - X \hat{\beta}_T\|_2^2 \leq \|y - X \beta\|_2^2$$ for all coefficients $\beta$. In particular, this holds for $\beta = \hat{\beta}_\textrm{GLS}$.

$\endgroup$
1
$\begingroup$

Hi: Maybe if you think of it this way it might might make more sense. By the gauss-markov theorem, when the assumptions of the theorem are satified, OLS is efficient. This means that the OLS estimate has the smallest possible variance of all linear unbiased estimators.

But, in the notes, author clearly shows that the GLS model can be transformed to an OLS model using the square root of the covariance matrix. See the section on Aitken.

So, if you view that that transformed model as an OLS model, which it is, then that model also has to satisfy the guass-markov theorem. ( assuming the assumptions hold. which they do ). So, it's variance has to be the smallest of all linear unbiased estimators also.

                                            Mark

Addendum: Although the above is true, it does NOT address What the OP was asking. Now that I understand what he-she is asking, I will think about his question and try to develop an answer but anyone is welcomed to also. My apologies to community for providing an answer that has very little to do with the question.

$\endgroup$
  • $\begingroup$ Thank you for your answer. However, you mention the variance of the estimator, and I cannot see the connection between the fact that the variance is lower and a higher SSE. $\endgroup$ – ahgert Sep 9 at 23:44
  • 1
    $\begingroup$ Actually, I went back to your notes and figured out what page 80 was. Now I realize that I made a mistake in interpreting your question. What you are asking is not what I was answering so let me delete my answer and comment and I ( or maybe someone else) can think about it some and give you a good answer. My apologies for the confusion. Originally, I couldn't find page 80 and just assumed that you were asking something that you weren't asking. My mistake. $\endgroup$ – mlofton Sep 11 at 3:17
  • $\begingroup$ Thank you very much. I believe Baer's answer is correct, since when calculating the sum of squared residuals we are not considering the matrix omega that is crucial when finding the GLS estimates. $\endgroup$ – ahgert Sep 11 at 18:01
  • $\begingroup$ A very nice answer by Baer. My intuition is that OLS is a projection so this is why it has to be a minimizer. Hopefully Baer or you can confirm that. So, the omega matrix works on the variance of the estimates but has to make a sacrifice by adding some bias. Thanks to Baer for great answer. $\endgroup$ – mlofton Sep 12 at 7:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.