18
$\begingroup$

Recently I was very embarrassed when I gave an off-the-cuff answer about minimum variance unbiased estimates for parameters of a uniform distribution that was completely wrong. Fortunately, I was immediately corrected by cardinal and Henry with Henry providing the correct answers for the OP.

This got me thinking though. I learned the theory of best unbiased estimators in my graduate math stat class at Stanford some 37 years ago. I have recollections of the Rao-Blackwell theorem, the Cramer - Rao lower bound and the Lehmann-Scheffe Theorem. But as an applied statistician I don't think very much about UMVUEs in my daily life whereas maximum likelihood estimation comes up a lot.

Why is that? Do we overemphasize the UMVUE theory too much in graduate school? I think so. First of all, unbiasedness is not a crucial property. Many perfectly good MLEs are biased. Stein shrinkage estimators are biased but dominate the unbiased MLE in terms of mean square error loss. It is a very beautiful theory (UMVUE estimation), but very incomplete and I think not very useful. What do others think?

Improve this question
$\endgroup$
7
  • 5
    $\begingroup$ (+1) I concur that this would make a good question for the main site and will upvote it. It is somewhat subjective, so it might be best as a CW question. (Also, there is no reason to be embarrassed.) $\endgroup$
    – cardinal
    Commented Aug 7, 2012 at 13:17
  • 2
    $\begingroup$ I do not think that, in general, this sort of estimation is overemphasised. I remember that my professors used to focus more on examples where UMVUE are "silly". People tend to use point estimators belonging to popular theories, for the sake of safety, but there is a complete theory of estimating equations. Some professors focus on UMVUE because they are a good source of difficult problems for homework. I think that bias reduction is a more popular and useful theory nowadays than finding the UMVUE (which not always exist). $\endgroup$
    – user10525
    Commented Aug 7, 2012 at 14:59
  • 2
    $\begingroup$ We see a lot of questions here on UMVUE I guess because they make good homework problems. Maybe this is more of a problem with undergraduate and masters level statistics programs than with PhD programs. $\endgroup$
    – Michael R. Chernick
    Commented Aug 7, 2012 at 15:08
  • 3
    $\begingroup$ Well, UMVU estimation is a classic idea so should maybe be teached for that reason? And it is a good starting point for discussing/critcizing criteria such as unbiasedness! Just because they are not so much used in practice, in itself is no reason to not teach them. $\endgroup$
    – kjetil b halvorsen ♦
    Commented Aug 8, 2012 at 4:21
  • 3
    $\begingroup$ The emphasis is likely to vary across time and departments. My department presents the material in the first year math stat course, but after that it is gone, so I couldn't reasonably say that it is overemphasized (even in the PhD inference course, it typically isn't taught, in favor of more time with Bayesian and minimax estimators, admissibility, and multivariate estimation), even though I wish there was more of an emphasis on why bias is a useful thing and hence why unbiased estimation is an unnecessarily extreme paradigm. $\endgroup$
    – guy
    Commented Aug 10, 2012 at 4:18

2 Answers 2

Reset to default
3
$\begingroup$

We know that

If $X_1,X_2, \dots X_n$ be a random sample from $Poisson(\lambda)$ then for any $\alpha \in (0,1),~T_\alpha =\alpha \bar X+(1-\alpha)S^2$ is an UE of $\lambda$

Hence there exists infinitely many UE's of $\lambda$. Now a question occur which of these should we choose? so we call UMVUE. Along unbiasedness is not a good property but UMVUE is a good property. But it is not extremely good.

If $X_1,X_2, \dots X_n$ be a random sample from $N(\mu, \sigma^2)$ then minimum MSE estimator of the form $T_\alpha =\alpha S^2$, with $(n-1)S^2=\sum_{i=1}^{n}(X_i-\bar X)^2$ for the parameter $\sigma^2$, is $\frac{n-1}{n+1}S^2=\frac{1}{n+1}\sum_{i=1}^{n}(X_i-\bar X)^2$ But it is biased that is it is not UMVUE though it is best in terms of minimum MSE.

Note that Rao-Blackwell Theorem says that to find UMVUE we can concentrate only on those UE which are function of sufficient statistic that is the UMVUE is the estimator which has minimum variance among all UEs which are function of sufficient statistic. Hence UMVUE is necessarily a function of a sufficient statistic.

MLE and UMVUE both are good from a point of view. But we can never say that one of them is better than other. In statistics we deal with uncertain and random data. So there is always scope for improvement. We may get a better estimator than MLE and UMVUE.

I think we don't overemphasize the UMVUE theory too much in graduate school.It is purely my personal view. I think graduation stage is a learning stage. So, a graduated student must need to carry a good basis about UMVUE and others estimators,

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ I think any valid theory of inference is good to know. While unbiasedness can be a good property, bias is not necessarily bad. When emphasis is put on UMVUEs there can be a tendency to attribute "optimality" to it. But there may not be any very good estimators in the class of unbiased estimators. Accuracy is important and it involves both bias and variance. What is better about the MLE is that there are conditions under which it can be shown to be asymptotically efficient. $\endgroup$
    – Michael R. Chernick
    Commented Aug 11, 2012 at 13:48
  • $\begingroup$ Note that Rao-Blackwell theorem can also be used to improve any biased estimator, producing an improved estimator with the same bias. $\endgroup$
    – kjetil b halvorsen ♦
    Commented Mar 27, 2017 at 20:00
2
$\begingroup$

Perhaps the paper by Brad Efron "Maximum Likelihood and Decision Theory" can help clarify this. Brad mentioned that one main difficulty with the UMVUE is that it is in general hard to compute, and in many cases do not exist.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.