1
$\begingroup$

I am following Linear Modfels in Statistics, Rencher & Schaalje, 2nd Edicition for the proof of Gauss-Markov Theorem (Theorem 7.3d, Page 146). I understand how least squares estimator $\mathbf{\hat{\beta}_{LS}} = (\mathbf{X}^{\mathsf{T}}\mathbf{X})^{-1}\mathbf{X}^{\mathsf{T}}\mathbf{y}$ achieves minimum variance among all linear unbiased estimators for $\beta_1, \ldots, \beta_p$. I am not able to figure out whether $\mathbf{\hat{\beta}_{LS}}$ is the only linear unbiased estimator which achieves this minimum variance for each $\beta_1, \ldots, \beta_p$.

$\endgroup$
1
  • $\begingroup$ Have you looked at the Wikipedia page, in particular remarks on proof section which seems to show uniqueness $\endgroup$ – seanv507 Jan 24 at 11:17
1
$\begingroup$

Well, there are other estimators, but they must by equal to $\hat{\boldsymbol\beta}_{LS}$ :)

If you look for $\arg\min_\beta(\mathbf{y}-\mathbf{X}\boldsymbol{\beta})^T(\mathbf{y}-\mathbf{X}\boldsymbol{\beta})$ then you find $\hat{\boldsymbol{\beta}}_{LS}$.

If you assume normality and look for a maximum likelihood estimator you get $\hat{\boldsymbol{\beta}}_{ML}=\hat{\boldsymbol{\beta}}_{LS}$. BTW, the Gauss-Markov theorem does not assume normality and states that $\hat{\boldsymbol{\beta}}_{LS}$ is the best linear unbiased estimator, but if you assume normality then $\hat{\boldsymbol{\beta}}_{ML}$ is the best - linear or nonlinear - unbiased estimator (e.g. Seber & Lee, Linear Regression Analysis, John Wiley & Sons, 2003, §3.5).

If you assume that $\mathbf{X}$ is stochastic, i.e. that the population model is $y=\mathbf{x}\boldsymbol\beta+\varepsilon$ where $\mathbf{x}$ is a random row vector, you can premultiply by $\mathbf{x}'$, take expectations assuming $E[\mathbf{x}'u]=\mathbf{0}$ and get $\boldsymbol\beta=E[\mathbf{x}'\mathbf{x}]^{-1}E[\mathbf{x}'y]$. Replacing the population moments $E[\mathbf{x}'\mathbf{x}]^{-1}$ and $E[\mathbf{x}'y]$ with the corresponding sample averages you get a method of moments estimator $\hat{\boldsymbol{\beta}}_{MM}=\hat{\boldsymbol{\beta}}_{LS}$ (e.g. Wooldridge, Econometric Analysis of Cross Section and Panel Data, MIT Press, 2010, §4.2.1).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.