In the standard proof of the Gauss-Markov theorem with $p$, an alternative linear estimator of the $p \times 1$ coefficient vector $\beta$ is $$ \tilde \beta = \hat \beta + \delta \tilde \beta = (X^T X)^{-1} X^T y + D^Ty $$ where $X$ and $D$ are $N \times p$ and $y$ is $N \times 1$. This has expectation $E[\tilde \beta] = \beta + D^TX\beta$. If the estimator is to be unbiased, $D^TX\beta = 0$. The immediate conclusion in the several sources I find is that $D^TX=0$. This is not a fact of linear algebra; all that is is required is that $D^TX$ have a null space that includes $\beta$. Nor am I sure that you argue that this must be true for arbitrary $\beta$, since $\beta$ is not arbitrary and the information available in $X$ and $y$ (of which $D$ is a function) is not independent of $\beta$. Instead, the best justification that I can make is that $D^TX$ cannot be a a matrix with a non-trivial null space that includes $\beta$, because this would give precise information about $\beta$ (e.g. that some linear combination $v \cdot \beta$ is exactly equal to zero), which is impossible. I believe this justification, while true, is not at all rigorous and may not be complete.

So, my question is: What is the formal argument that $D^T X \beta = 0 \Rightarrow D^T X = 0$? Perhaps I am incorrect that you cannot argue $\forall \beta \in \mathbb{R}^p, \, D^T X \beta = 0 \Rightarrow D^T X = 0$? An ill-defined thought: maybe conditioning $\forall \beta$ doesn't require assume that any $\beta$ is equally likely...

  • 1
    $\begingroup$ This actually is just a linear algebra fact. The estimator has to work for all possible $\beta$. You don't have to assume any probability distribution for those $\beta$ either. It suffices that you simply don't know what $\beta$ actually is, implying that any property you desire of your estimator must be one that holds for all possible such values. $\endgroup$ – whuber Oct 17 '16 at 22:32
  • 1
    $\begingroup$ @whuber Thank you; I understand now. I think I had it in my head that $D$ could depend on $X$ and $y$ (in which case it would have implicit dependence on $\beta$), but of course that is excluded by the restriction to linear estimators -- $D$ can only depend on $X$, and the entire argument of the proof is conditional on fixed $X$. (And $E[D^Ty] \neq D^T X \beta$ if D is a function of $y$, as well). $\endgroup$ – jwimberley Oct 18 '16 at 13:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.