1
$\begingroup$

enter image description here

Hi, given this information we are meant to prove that the above estimator is unbiased. I understand the proof for the most part (below). What I do not understand is the intuitive reason why the projection matrix, or Mx, is in this proof. Can someone explain this to me? Also is there another way to show this estimator is unbiased?

Thanks!

enter image description here

$\endgroup$
  • $\begingroup$ It is difficult to see what you are asking: isn't $M_x$ part of the definition of $\hat\varepsilon$? Isn't that a more than sufficient reason why it should appear in the proof? $\endgroup$ – whuber Feb 19 '16 at 21:59
1
$\begingroup$

Note 1: $M_X$ is not the projection matrix, but is called the residual maker for reasons I will make obvious below. Define $P_X = X(X'X)^{-1}X'$ as the projection matrix (into the column space of matrix $X$). Then, $M_X = (I-P_X)$. Also note that both $P_X$ and $M_X$ are symmetric and idempotent (i.e., $P_X'P_X = P_XP_X =P_X$).

Note 2: By definition, $\hat{\varepsilon} = Y - X\hat{\beta}$, where $\hat{\beta}$ denotes the OLS estimator of $\beta$. Now note that if we suppose that $Y = X\beta + \varepsilon$, then by how we have defined $M_X$, $M_XY = (Y - P_XY) = (Y - X\hat{\beta}) = \hat{\varepsilon}$. This is why we call $M_X$ residual maker in the first place.

Note 3: Now comes the interesting observation: By definition, ${\varepsilon} = Y - X{\beta}$. Thus, $M_X\varepsilon = M_X(X\beta - Y) = M_XX\beta - M_XY \overset{Note 2}{=} M_XX\beta - \hat{\varepsilon}$. Now all that is left to show is that $M_XX\beta = 0$, which is straightforward: $M_XX\beta = (I-P_X)X\beta = X\beta - P_XX\beta = X\beta - X(X'X)^{-1}X'X\beta = X\beta - X\beta = 0$.

Note 4: The intuitive reason that we need $M_X$ is that whilst ${\varepsilon}$ is unobservable, $\hat{\varepsilon}$ and $M_X$ are observables, so we can relate an unobservable quantity of interest to two observable quantities.

I hope this can help you out :)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.