I am following Linear Modfels in Statistics, Rencher & Schaalje, 2nd Edicition for the proof of Gauss-Markov Theorem (Theorem 7.3d, Page 146). I understand how least squares estimator $\mathbf{\hat{\beta}_{LS}} = (\mathbf{X}^{\mathsf{T}}\mathbf{X})^{-1}\mathbf{X}^{\mathsf{T}}\mathbf{y}$ achieves minimum variance among all linear unbiased estimators for $\beta_1, \ldots, \beta_p$. I am not able to figure out whether $\mathbf{\hat{\beta}_{LS}}$ is the only linear unbiased estimator which achieves this minimum variance for each $\beta_1, \ldots, \beta_p$.
$\begingroup$
$\endgroup$
1
-
$\begingroup$ Have you looked at the Wikipedia page, in particular remarks on proof section which seems to show uniqueness $\endgroup$ – seanv507 Jan 24 at 11:17
Add a comment
|
$\begingroup$
$\endgroup$
Well, there are other estimators, but they must by equal to $\hat{\boldsymbol\beta}_{LS}$ :)
If you look for $\arg\min_\beta(\mathbf{y}-\mathbf{X}\boldsymbol{\beta})^T(\mathbf{y}-\mathbf{X}\boldsymbol{\beta})$ then you find $\hat{\boldsymbol{\beta}}_{LS}$.
If you assume normality and look for a maximum likelihood estimator you get $\hat{\boldsymbol{\beta}}_{ML}=\hat{\boldsymbol{\beta}}_{LS}$. BTW, the Gauss-Markov theorem does not assume normality and states that $\hat{\boldsymbol{\beta}}_{LS}$ is the best linear unbiased estimator, but if you assume normality then $\hat{\boldsymbol{\beta}}_{ML}$ is the best - linear or nonlinear - unbiased estimator (e.g. Seber & Lee, Linear Regression Analysis, John Wiley & Sons, 2003, §3.5).
If you assume that $\mathbf{X}$ is stochastic, i.e. that the population model is $y=\mathbf{x}\boldsymbol\beta+\varepsilon$ where $\mathbf{x}$ is a random row vector, you can premultiply by $\mathbf{x}'$, take expectations assuming $E[\mathbf{x}'u]=\mathbf{0}$ and get $\boldsymbol\beta=E[\mathbf{x}'\mathbf{x}]^{-1}E[\mathbf{x}'y]$. Replacing the population moments $E[\mathbf{x}'\mathbf{x}]^{-1}$ and $E[\mathbf{x}'y]$ with the corresponding sample averages you get a method of moments estimator $\hat{\boldsymbol{\beta}}_{MM}=\hat{\boldsymbol{\beta}}_{LS}$ (e.g. Wooldridge, Econometric Analysis of Cross Section and Panel Data, MIT Press, 2010, §4.2.1).
