4
$\begingroup$

I checked all the books and on-line materials I could find for the proof, but found all of them have a derivation problem, which I cannot understand.

To prove the least squares estimator is the $BLUE$ for the linear model $y = X*\beta + v$, one assumes $c = C*y$ is any linear unbiased estimator of $\beta$. Using the fact that $c$ is an unbiased estimator, we can easily obtain $(C*X-I)\beta = 0$.

Then all the books and on-line materials just conclude that $C*X-I$ must be 0. I don't understand this at all. If the equality $(C*X-I)\beta = 0$ should hold for any $\beta$, then certainly we would have $C*X-I = 0$. But I don't know any reason that $(C*X-I)\beta$ = 0 should hold for any $\beta$. Is there some one who can explain this to me?

$\endgroup$
1
  • $\begingroup$ When you are doing regression, you don't know what $\beta$ is. If you want your procedure to be BLUE, then, it must be BLUE regardless of the true value of $\beta$. It's not enough for it to be BLUE for just some $\beta$ (such as your particular one) because even when you have made your estimate, you still don't know what the true $\beta$ is. $\endgroup$
    – whuber
    Mar 29, 2017 at 16:10

1 Answer 1

3
$\begingroup$

The OLS estimator is $$ \hat{\boldsymbol{\beta}} = (\mathbf{X}'\mathbf{X})^{-1}\mathbf{X}'\boldsymbol{Y} $$

The class of linear unbiased estimators is $\tilde{\boldsymbol{\beta}} = \mathbf{C}\boldsymbol{Y}$, for $\mathbf{C} = \mathbf{f}(\mathbf{X})$ (that is $\mathbf{C}$ is a matrix valued function of $\mathbf{X}$), such that $$ \begin{align} \mathbb{E}(\tilde{\boldsymbol{\beta}} \mid \mathbf{X}) &= \boldsymbol{\beta} \\ \mathbf{C}\mathbb{E}(\boldsymbol{Y} \mid \mathbf{X}) &= \boldsymbol{\beta} \\ \mathbf{C}\mathbf{X}\boldsymbol{\beta} &=\boldsymbol{\beta} \end{align} $$ where the last step follows from the linear regression model $\mathbb{E}(\boldsymbol{Y} \mid \mathbf{X}) = \mathbf{X}\boldsymbol{\beta}$.

Hence, it follows that $$ \mathbf{C}\mathbf{X} = \boldsymbol{\iota}_K $$ where $\boldsymbol{\iota}_K$ is the identity matrix of size $K$.

$\endgroup$
7
  • $\begingroup$ Yes, CX*\beta should be equal to \beta, but why should CX be equal to I ? $\endgroup$
    – Alan
    Feb 17, 2014 at 18:48
  • $\begingroup$ @Alan From the definition of the identity matrix. $\boldsymbol{\iota}$ is the identity matrix if $\forall$ matrices $\mathbf{a}$, $\boldsymbol{\iota}\mathbf{a} = \mathbf{a}$. $\endgroup$ Feb 17, 2014 at 18:50
  • $\begingroup$ My question was why should (CX-I)*\beta=0 hold for any \beta? Can not \beta be a fixed parameter vector? $\endgroup$
    – Alan
    Feb 17, 2014 at 18:56
  • $\begingroup$ @Alan What difference does that make? $\endgroup$ Feb 17, 2014 at 19:17
  • 1
    $\begingroup$ Yes, this should hold for any unbiased estimator $\tilde{\beta}$ --- different $C$ gives different estimator. But $\beta$ in $(CX-I)\beta=0$ is the true parameter vector, not an estimator. $\endgroup$
    – Alan
    Feb 17, 2014 at 21:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.