This question was raised before. But so far I haven't got any satisfactory answer yet. So I raise it again.

Suppose we have the linear model $$ y = X \beta + v, \quad v \sim (0, \sigma^2 I), $$ where $X$ has full column rank. The least squares estimator $\hat{\beta}=X^\dagger y$ is claimed to be the best linear unbiased estimator of the true parameter vector $\beta$. I checked all the books and on-line materials I could find for the proof, but found all of them seem to have a derivation problem.

To prove it, one assumes $\bar{\beta} = Cy$ is any linear unbiased estimator of $\beta$. Using the fact that $\bar{\beta}$ is an unbiased estimator, we can easily obtain $(CX-I)\beta = 0$. Then all the books and on-line materials just conclude that $CX-I$ must be 0. I don't understand this at all. If $\beta$ were arbitrary, then certainly we would have $CX-I = 0$. But $\beta$ here is a specific parameter vector. Why can one claim $CX-I=0$ ?

  • 1
    $\begingroup$ How would you propose exploiting the fact that $\beta$ is "specific"? (The point behind this question is that an estimator is a procedure, not just the value you happen to obtain in any particular circumstance, and so properties like "best" and "unbiased" have to be properties of the procedure regardless of the value $\beta$ happens to take.) $\endgroup$ – whuber Feb 26 '14 at 19:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.