22
$\begingroup$

I know for regular problems, if we have a best regular unbiased estimator, it must be the maximum likelihood estimator (MLE). But generally, if we have an unbiased MLE, would it also be the best unbiased estimator (or maybe I should call it UMVUE, as long as it has the smallest variance)?

$\endgroup$
  • 3
    $\begingroup$ Interesting question. MLE is a function of the sufficient statistic, and UMVUEs can be obtained by conditioning on complete and sufficient statistics. So if MLE is unbiased (and a function of the sufficient statistic), the only way possible for it to not have minimum variance is if the sufficient statistic is not complete. I tried to find an example, but was unsuccessful. $\endgroup$ – Greenparker Apr 30 '16 at 20:52
  • 2
    $\begingroup$ And here is some brief information about sufficient and complete statistics. $\endgroup$ – Richard Hardy May 1 '16 at 8:09
  • 10
    $\begingroup$ The real issue is more that the MLE is rarely unbiased: if $\theta$ is the unbiased estimator of $\theta$ and the MLE of $\theta$, $f(\hat\theta)$ is the MLE of $f(\theta)$ but is biased for most bijective transforms $f$. $\endgroup$ – Xi'an May 2 '16 at 7:01
  • 1
    $\begingroup$ Is this relevant? "An almost unbiased estimator of population mean" Vyas Dubey Pt.Ravishankar Shukla University, Raipur, India $\endgroup$ – user114445 May 3 '16 at 20:24
  • 2
    $\begingroup$ +1 for Xi'ans comment. Best estimator means minimal variance, unbiased means something else. So I'm not sure that you can start trying to prove that, since one has little to do with the other. But before I'd even start my own derivation, I'd like to see some serious effort in the (try of a) proof. I'd say that even the proof of the first statement (MLE is optimal for certain cases) is not trivial. $\endgroup$ – cherub Jan 8 '18 at 12:52
13
$\begingroup$

In my opinion, the question is not truly coherent in that the maximisation of a likelihood and unbiasedness do not get along, if only because maximum likelihood estimators are equivariant, ie the transform of the estimator is the estimator of the transform of the parameter, while unbiasedness does not stand under non-linear transforms. Therefore, maximum likelihood estimators are almost never unbiased, if "almost" is considered over the range of all possible parametrisations.

However, there is an more direct answer to the question: when considering the estimation of the Normal variance, $\sigma^2$, the UMVUE of $\sigma^2$ is $$\hat{\sigma}^2_n = \frac{1}{n-1} \sum_{i=1}^n \{x_i-\bar{x}_n\}^2$$ while the MLE of $\sigma^2$ is $$\check{\sigma}^2_n = \frac{1}{n} \sum_{i=1}^n \{x_i-\bar{x}_n\}^2$$ Ergo, they differ. This implies that

if we have a best regular unbiased estimator, it must be the maximum likelihood estimator (MLE).

does not hold in general.

Note further that, even when there exist unbiased estimators of a parameter $\theta$, there is no necessarily a best unbiased minimum variance estimator (UNMVUE).

$\endgroup$
  • $\begingroup$ So can we say that an unbiased MLE it is a (U)MVUE, but not every (U)MVUE is the MLE? $\endgroup$ – Sextus Empiricus Jan 9 '18 at 13:14
  • 2
    $\begingroup$ No, we have no reason to believe this is true in general. $\endgroup$ – Xi'an Jan 9 '18 at 13:57
13
+50
$\begingroup$

But generally, if we have an unbiased MLE, would it also be the best unbiased estimator ?

If there is a complete sufficient statistics, yes.

Proof:

  • Lehmann–Scheffé theorem: Any unbiased estimator that is a function of a complete sufficient statistics is the best (UMVUE).
  • MLE is a function of any sufficient statistics. See 4.2.3 here;

Thus an unbiased MLE is necesserely the best as long as a complete sufficient statistics exists.

But actually this result has almost no case of application since a complete sufficient statistics almost never exists. It is because complete sufficient statistics exist (essentially) only for exponential families where the MLE is most often biased (except location parameter of Gaussians).

So the real answer is actually no.

A general counter example can be given: any location family with likelihood $p_\theta(x)=p(x-\theta$) with $p$ symmetric around 0 ($\forall t\in\mathbb{R} \quad p(-t)=p(t)$). With sample size $n$, the following holds:

  • the MLE is unbiased
  • it is dominated by another unbiased estimator know as Pitman's equivariant estimator

Most often the domination is strict thus the MLE is not even admissible. It was proven when $p$ is Cauchy but I guess it's a general fact. Thus MLE can't be UMVU. Actually, for these families it's known that, with mild conditions, there is never an UMVUE. The example was studied in this question with references and a few proofs.

$\endgroup$
  • $\begingroup$ Why doesn't this have the highest upvotes? I felt this answer was better than Xian's. $\endgroup$ – Red Floyd Nov 16 '18 at 15:10
0
$\begingroup$

MLE's asymptotic variance is UMVUE i.e attains cramer rao lower bound but finite variance may not be UMVUE to make sure that estimator is UMVUE it should be sufficient and complete statistics or any function of that statistics.

$\endgroup$
0
$\begingroup$

In short, an estimator is UMVUE, if it is unbiased and the function of a complete and sufficient statistic. (See Rao-Blackwell and Scheffe)

$\endgroup$
  • $\begingroup$ Which means this is restricted to exponential families. $\endgroup$ – Xi'an Jan 15 '18 at 11:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.