11
$\begingroup$

I'm reading up on the Guass-Markov theorem on wikipedia, and I was hoping somebody could help me figure out the main point of the theorem.

We assume a linear model, in matrix form, is given by: $$ y = X\beta +\eta $$ and we're looking for the BLUE, $ \widehat\beta $.

In accordance with this, I would label $\eta = y - X\beta$ the "residual" and $\varepsilon = \widehat\beta - \beta$ the "error". (I.e the opposite of the usage on the Gauss-Markov page).

The OLS (ordinary least-squares) estimator may be derived as the argmin of $||\text{residual}||_2^2 = ||\eta||_2^2$.

Now, let $\mathbb{E}$ denote the expectation operator. To my understanding, what the Gauss-Markov theorem tells us is that, if $\mathbb{E}(\eta) = 0$ and $\text{Var}(\eta) = \sigma^2 I $, then the argmin, over all linear, unbiased estimators, of $\mathbb{E}(||\text{error}||_2^2) = \mathbb{E} (||\varepsilon||_2^2)$ is given by the same expression as the OLS estimator.

I.e. $$ \text{argmin}_{\text{} \widehat\beta(y)} \, ||\eta||_2^2 \;=\; (X'X)^{-1}X'y \;=\; \text{argmin}_{\text{linear, unbiased } \widehat\beta(y)} \, \mathbb{E}(||\varepsilon||_2^2)$$

Is my understanding correct? And if so, would you say that it deserves more prominent emphasis in the article?

$\endgroup$

2 Answers 2

15
$\begingroup$

I am not sure if I understood you question correctly, but if you are looking to prove that the OLS for $\hat{\beta}$ is BLUE (best linear unbiased estimator) you have to prove the following two things: First that $\hat{\beta}$ is unbiased and second that $Var(\hat{\beta})$ is the smallest among all linear unbiased estimators.

Proof that OLS estimator is unbiased can be found here http://economictheoryblog.com/2015/02/19/ols_estimator/

and proof that $Var(\hat{\beta})$ is the smallest among all linear unbiased estimators can be found here http://economictheoryblog.com/2015/02/26/markov_theorem/

$\endgroup$
1
  • $\begingroup$ Proofs are helpful, yes. $\endgroup$
    – Patrick
    Feb 27, 2015 at 13:30
0
$\begingroup$

It seems my hunch was correct indeed, as confirmed, e.g. on page 375 of the book Introductory Econometrics. Relevant excerpt:

Excerpt from book

$\endgroup$
2
  • $\begingroup$ Please write something more since your answer could be helpful for others in the future. $\endgroup$
    – Tim
    Feb 27, 2015 at 7:24
  • $\begingroup$ Your link is broken. $\endgroup$
    – Glen_b
    Mar 5, 2015 at 1:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.