I know for regular problems, if we have a best regular unbiased estimator, it must be the maximum likelihood estimator (MLE). But generally, if we have an unbiased MLE, would it also be the best unbiased estimator (or maybe I should call it UMVUE, as long as it has the smallest variance)?
$\begingroup$
$\endgroup$
6
-
4$\begingroup$ Interesting question. MLE is a function of the sufficient statistic, and UMVUEs can be obtained by conditioning on complete and sufficient statistics. So if MLE is unbiased (and a function of the sufficient statistic), the only way possible for it to not have minimum variance is if the sufficient statistic is not complete. I tried to find an example, but was unsuccessful. $\endgroup$– GreenparkerCommented Apr 30, 2016 at 20:52
-
2$\begingroup$ And here is some brief information about sufficient and complete statistics. $\endgroup$– Richard HardyCommented May 1, 2016 at 8:09
-
11$\begingroup$ The real issue is more that the MLE is rarely unbiased: if $\theta$ is the unbiased estimator of $\theta$ and the MLE of $\theta$, $f(\hat\theta)$ is the MLE of $f(\theta)$ but is biased for most bijective transforms $f$. $\endgroup$– Xi'anCommented May 2, 2016 at 7:01
-
1$\begingroup$ Is this relevant? "An almost unbiased estimator of population mean" Vyas Dubey Pt.Ravishankar Shukla University, Raipur, India $\endgroup$– user114445Commented May 3, 2016 at 20:24
-
2$\begingroup$ +1 for Xi'ans comment. Best estimator means minimal variance, unbiased means something else. So I'm not sure that you can start trying to prove that, since one has little to do with the other. But before I'd even start my own derivation, I'd like to see some serious effort in the (try of a) proof. I'd say that even the proof of the first statement (MLE is optimal for certain cases) is not trivial. $\endgroup$– cherubCommented Jan 8, 2018 at 12:52
|
Show 1 more comment
4 Answers
$\begingroup$
$\endgroup$
1
But generally, if we have an unbiased MLE, would it also be the best unbiased estimator ?
If there is a complete sufficient statistics, yes.
Proof:
- Lehmann–Scheffé theorem: Any unbiased estimator that is a function of a complete sufficient statistics is the best (UMVUE).
- MLE is a function of any sufficient statistics. See 4.2.3 here;
Thus an unbiased MLE is necesserely the best as long as a complete sufficient statistics exists.
But actually this result has almost no case of application since a complete sufficient statistics almost never exists. It is because complete sufficient statistics exist (essentially) only for exponential families where the MLE is most often biased (except location parameter of Gaussians).
So the real answer is actually no.
A general counter example can be given: any location family with likelihood $p_\theta(x)=p(x-\theta$) with $p$ symmetric around 0 ($\forall t\in\mathbb{R} \quad p(-t)=p(t)$). With sample size $n$, the following holds:
- the MLE is unbiased
- it is dominated by another unbiased estimator know as Pitman's equivariant estimator
Most often the domination is strict thus the MLE is not even admissible. It was proven when $p$ is Cauchy but I guess it's a general fact. Thus MLE can't be UMVU. Actually, for these families it's known that, with mild conditions, there is never an UMVUE. The example was studied in this question with references and a few proofs.
-
1$\begingroup$ Why doesn't this have the highest upvotes? I felt this answer was better than Xian's. $\endgroup$ Commented Nov 16, 2018 at 15:10
$\begingroup$
$\endgroup$
2
In my opinion, the question is not truly coherent in that the maximisation of a likelihood and unbiasedness do not get along, if only because maximum likelihood estimators are equivariant, ie the transform of the estimator is the estimator of the transform of the parameter, while unbiasedness does not stand under non-linear transforms. Therefore, maximum likelihood estimators are almost never unbiased, if "almost" is considered over the range of all possible parametrisations.
However, there is a more direct answer to the question: when considering the estimation of the Normal variance, $\sigma^2$, the UMVUE of $\sigma^2$ is $$\hat{\sigma}^2_n = \frac{1}{n-1} \sum_{i=1}^n \{x_i-\bar{x}_n\}^2$$ while the MLE of $\sigma^2$ is $$\check{\sigma}^2_n = \frac{1}{n} \sum_{i=1}^n \{x_i-\bar{x}_n\}^2$$ Ergo, they differ. This implies that
if we have a best regular unbiased estimator, it must be the maximum likelihood estimator (MLE).
does not hold in general.
Note further that, even when there exist unbiased estimators of a parameter $\theta$, there is no necessarily a best unbiased minimum variance estimator (UNMVUE).
-
$\begingroup$ So can we say that an unbiased MLE it is a (U)MVUE, but not every (U)MVUE is the MLE? $\endgroup$ Commented Jan 9, 2018 at 13:14
-
2$\begingroup$ No, we have no reason to believe this is true in general. $\endgroup$– Xi'anCommented Jan 9, 2018 at 13:57
$\begingroup$
$\endgroup$
MLE's asymptotic variance is UMVUE i.e attains cramer rao lower bound but finite variance may not be UMVUE to make sure that estimator is UMVUE it should be sufficient and complete statistics or any function of that statistics.
$\begingroup$
$\endgroup$
1
In short, an estimator is UMVUE, if it is unbiased and the function of a complete and sufficient statistic. (See Rao-Blackwell and Scheffe)
-
$\begingroup$ Which means this is restricted to exponential families. $\endgroup$– Xi'anCommented Jan 15, 2018 at 11:03