Is unbiased maximum likelihood estimator always the best unbiased estimator? I know for regular problems, if we have a best regular unbiased estimator, it must be the maximum likelihood estimator (MLE). But generally, if we have an unbiased MLE, would it also be the best unbiased estimator (or maybe I should call it UMVUE, as long as it has the smallest variance)?
 A: 
But generally, if we have an unbiased MLE, would it also be the best
  unbiased estimator ?

If there is a complete sufficient statistics, yes. 
Proof:


*

*Lehmann–Scheffé theorem: Any unbiased estimator that is a function of a complete sufficient statistics 
is the best (UMVUE).

*MLE is a function of any sufficient statistics. See 4.2.3 here;


Thus an unbiased MLE is necesserely the best as long as a complete sufficient statistics exists.
But actually this result has almost no case of application since a complete sufficient statistics almost never exists. It is because complete sufficient statistics exist (essentially) only for exponential families where the MLE is most often biased (except location parameter of Gaussians).
So the real answer is actually no.
A general counter example can be given: any location family with likelihood $p_\theta(x)=p(x-\theta$) with $p$ symmetric around 0 ($\forall t\in\mathbb{R} \quad p(-t)=p(t)$). With sample size $n$, the following holds:


*

*the MLE is unbiased

*it is dominated by another unbiased estimator know as Pitman's equivariant estimator


Most often the domination is strict thus the MLE is not even admissible. It was proven when $p$ is Cauchy but I guess it's a general fact. Thus MLE can't be UMVU. Actually, for these families it's known that, with mild conditions, there is never an UMVUE. The example was studied in this question with references and a few proofs.
A: In my opinion, the question is not truly coherent in that the maximisation of a likelihood and unbiasedness do not get along, if only because maximum likelihood estimators are equivariant, ie the transform of the estimator is the estimator of the transform of the parameter, while unbiasedness does not stand under non-linear transforms. Therefore, maximum likelihood estimators are almost never unbiased, if "almost" is considered over the range of all possible parametrisations.
However, there is a more direct answer to the question: when considering the estimation of the Normal variance, $\sigma^2$, the UMVUE of $\sigma^2$ is
$$\hat{\sigma}^2_n = \frac{1}{n-1} \sum_{i=1}^n \{x_i-\bar{x}_n\}^2$$
while the MLE of $\sigma^2$ is
$$\check{\sigma}^2_n = \frac{1}{n} \sum_{i=1}^n \{x_i-\bar{x}_n\}^2$$
Ergo, they differ. This implies that

if we have a best regular unbiased estimator, it must be the maximum
likelihood estimator (MLE).

does not hold in general.
Note further that, even when there exist unbiased estimators of a parameter $\theta$, there is no necessarily a best unbiased minimum variance estimator (UNMVUE).
A: MLE's asymptotic variance is UMVUE i.e attains cramer rao lower bound but finite variance may not be UMVUE to make sure that estimator is UMVUE it should be sufficient and complete statistics or any function of that statistics.
A: In short, an estimator is UMVUE, if it is unbiased and the function of a complete and sufficient statistic. (See Rao-Blackwell and Scheffe)
