11
$\begingroup$

Consider an ordinary least squares model, $$y = \beta X + \epsilon \qquad \epsilon\sim N(0, \sigma)$$

The Gauss-Markov theorem tells us that the ordinary least-squares (OLS) estimator is the minimum-variance linear unbiased estimator (BLUE) for the coefficients: $$ \beta \approx \hat\beta = (X^TX)^{-1}X^Ty $$

Does an unbiased, nonlinear estimator with lower variance, $\tilde\beta$, exist?

Based on my previous question.

$\endgroup$
3
  • $\begingroup$ Not under normality. If the error is e.g. Laplace distributed, the mean absolute deviation estimator is more efficient. $\endgroup$
    – hejseb
    Jul 4 '17 at 4:45
  • $\begingroup$ Can you suggest why no such estimator $\tilde\beta$ exists? $\endgroup$
    – user126350
    Jul 4 '17 at 4:49
  • 10
    $\begingroup$ If the underlying data generating process is i.i.d gaussian with a constant variance, and a linear mean model (basically the model matches the truth), in that case OLS is the minimum variance unbiased estimator, because it attains the Cramer Rao Lower Bound. Note that I didn't say minimum variance linear unbiased. Basically it is the best. econ.ohio-state.edu/dejong/note5.pdf. page 17. $\endgroup$ Jul 4 '17 at 6:24
1
$\begingroup$

The Gauss-Markov theorem gives the conditions where the OLS estimator is the BLUE, and those conditions do not include normality of the residuals. When we also include that normality assumption, then we can remove the "L" and wind up with the "Best Unbiased Estimator", not just the best linear unbiased estimator (section 2.1, example 1 of the Ohio State econometrics notes).

However, if we do not make the normality assumption, then we can wind up with nonlinear estimators of the coefficients that have lower variance than the OLS estimate but are unbiased. For example, consider heavy-tailed errors and the solution given by minimizing absolute loss (quantile regression at the median), as I do here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.