2
$\begingroup$

Hi i am stuck on this one, the question is related to Gauss-Markov theorem:

Consider a general alternative to the OLS estimator that is also a linear unbiased estimator, say ${\tilde \beta}$. Outline a proof that the OLS estimator b is better, in a well defined sense, than ${\tilde \beta}$.

For both estimators b and ${\tilde \beta}$ i have to use the fact that $E(UU') = \sigma^2I$.

I have shown already that $ b= \left(\mathbf {X}'\mathbf {X}\right)^{-1}\mathbf {X}'\mathbf {y} = \left(\mathbf {X'}\mathbf {X}\right)^{-1}\mathbf X'(\mathbf {X}\mathbf {\beta}+u)$

$=\left(\mathbf {X'}\mathbf {X}\right)^{-1}\mathbf X'\mathbf {X}\mathbf \beta+(X'X)^{-1}X'u$

$=\beta+(X'X)^{-1}X'u$

Theorem proved for linearity, now

$E(b)= E[ \beta+(X'X)^{-1}X'u]=\beta $ as $E[u]=0$

Theorem proved for Unbiasedness.

Variance-covariance matrix of b: $Var(b)=E(b-\beta)(b-\beta)'$ using again $E(UU') = \sigma^2I$

$=E(b-\beta)(b-\beta)'=\sigma^2(X'X)^{-1}$

in order to show that b is a better estimator that ${\tilde \beta}$ i need to follow the same reasoning still using the fact that $E(UU') = \sigma^2I$ and then conclude that b has the minimum variance of all estimators thus it is the best, that is

$Var(b)\leq Var(\tilde \beta)$ i guess.

I am not sure how to properly compare the two estimators so if someone could give me a hint it would be much appreciated.

$\endgroup$
4
  • 1
    $\begingroup$ The estimators will be considered in terms of their variance-covariance matrices and the difference (competitor's (-) ols') should be a positive semi-definite matrix. You have to find a way to express the competitor estimator in connection to OLS (hint: both are linear functions of the data (y). $\endgroup$ Commented Jan 6, 2015 at 18:42
  • $\begingroup$ thanks Alecos, i actually found it using $\tilde \beta \equiv (A+C)y = ((X'X)^{-1}X'+ C)y$ $\endgroup$
    – user304663
    Commented Jan 6, 2015 at 18:53
  • $\begingroup$ Exactly. But please post the solution as an answer here, so that the thread becomes useful to other people also. $\endgroup$ Commented Jan 6, 2015 at 19:07
  • $\begingroup$ If you're 'stuck on a question', this should probably be tagged self-study. See the tag wiki $\endgroup$
    – Glen_b
    Commented Feb 26, 2015 at 20:55

1 Answer 1

7
$\begingroup$

We can compute the variance-covariance matrice of b and $\tilde \beta$ and hence compare their variances in order to tell which one has the smallest variance. (Best estimator)

Knowing tha $E(U)=0$

$E(b-\beta)(b-\beta)'= E(\beta+(X'X)^{-1}X'u-\beta)(\beta+(X'X)^{-1}X'u-\beta)'$

$=E[(X'X)^{-1}X'u][(X'X)^{-1}X'u]$

$=E[(X'X)^{-1}X'uu'X(X'X)^{-1})$ recall that $E(uu')= \sigma^2I$

$=\sigma^2(X'X)^{-1}X'IX(X'X)^{-1}$

$=\sigma^2(X'X)^{-1}$

So it has been shown that b is unbiased as $E(b_i)=\beta_i$ and therefore $E(b)=\beta$ but also linear as $b=(X'X)^{-1}X'y$, $b=Ay$ with $A=(X'X)^{-1}X'$

Let $\tilde \beta$ a general linear unbiased estimator, $\tilde \beta\equiv(A+C)y=[(X'X)^{-1}X'+C)y$

$=[(X'X)^{-1}X'+C)(X\beta+u)$

$=\beta+(X'X)^{-1}X'u+CX\beta+Cu$, we require $CX=0$ $\forall$ $X$ for unbiasedness

$E(\tilde \beta)=E[\beta+(X'X)^{-1}Xu + CX\beta+Cu]$ with $CX=0$ required and $E(u)=0$

$=E(\beta)$

Hence $\tilde \beta$ is also unbiased and $\tilde \beta=b+Cy$ is linear

Variance-covariance matrix of $\tilde \beta$: $E(\tilde \beta-\beta)(\tilde \beta-\beta)'=E[(X'X)^{-1}X'u+Cu)[(X'X)^{-1}Xu+Cu)']$

$=E[(X'X)^{-1}X'uu'X(X'X)^{-1}+(X'X)^{-1}X'uu'C']+Cuu'X(X'X)^{-1}+Cuu'C'$

using the fact that $E(uu')=\sigma^2I$ which simplifies to

$E(\tilde \beta-\beta)(\tilde \beta-\beta)'= \sigma^2(X'X)^{-1}+\sigma^2(X'X)^{-1}X'C'+\sigma^2CX(X'X)^{-1}+\sigma^2CC'$

we know that $CX=0$ hence $(CX)'=X'C'=0$ so we have

$E(\tilde \beta-\beta)(\tilde \beta-\beta)'=\sigma^2(X'X)^{-1}+\sigma^2CC'$

$=\sigma^2[(X'X)^{-1}+CC']$ , Recall that $E(b-\beta)(b -\beta)'=\sigma^2(X'X)^{-1}$

Thus $E(\tilde \beta-\beta)(\tilde \beta-\beta)'=E(b-\beta)(b -\beta)'+ \sigma^2CC'$

We can conclude that $Var(b)\leq Var(\tilde \beta)$ which means that b is the best estimator.

$\endgroup$
2
  • $\begingroup$ (+1) Thanks for contributing. I would just add for clarity the obvious, that the result holds because $CC'$ is positive semidefinite matrix. $\endgroup$ Commented Jan 7, 2015 at 1:54
  • 1
    $\begingroup$ I had to solve the same problem at University lately and found your answer difficult to go through. Here you will find a simpler approach leading to the same result. economictheoryblog.com/2015/02/26/markov_theorem $\endgroup$
    – ProofGauss
    Commented Feb 26, 2015 at 16:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.