Sufficient conditions for asymptotic efficiency of MLE

Question

Maximum-likelihood estimators are, according to Wikipedia, asymptotically efficient, that is they achieve the Cramér-Rao bound when sample size tends to infinity. But this seems to require some regularity conditions.

Can you give, or point me to, a set of conditions that ensure that the maximum-likelihood estimator is asymptotically efficient? Are there different sets of conditions for this? Do the conditions change depending in whether the parameter to be estimated is scalar or multi-dimensional?

Answer 1

A fundamental condition for MLE-efficiency is correct specification. So this is not quasi-MLE territory.

1. Denote $s$ to be the score of the likelihood, and $H$ to be its Hessian.
2. The variance of MLE is, to begin with, $$V_{MLE} = \big[-E(H)\big]^{-1}\big[E(ss')\big]\big[-E(H)\big]^{-1}. \tag{1}$$

2. But under correct specification (and some regularity conditions on the density) the Information Equality holds and we have $$\big[-E(H)\big]^{-1} = \big[E(ss')\big] \tag{2}$$

3. It follows that the variance of the MLE becomes $$V_{MLE} = -\big[E(H)\big]^{-1} \tag{3}$$

4. But the RHS of $(3)$ is the Cramer-Rao bound for unbiased (or asymptotically unbiased) estimators. So MLE attains the Cramer-Rao bound.

See Amemiya, T. (1985). Advanced econometrics. Harvard University Press. ch. 1.3.2 and 4.2.4

Also, Lehmann, E. L.; Casella, G. (1998). Theory of Point Estimation (2nd ed.). Springer. p. 115 leading to Lemma 5.3 that is in page 116.

Answer 2

If you add the assumption that the criterion function you are maximising or estimating equation you are solving actually specifies the MLE (you're maximising the likelihood or solving the score equations) then any of the standard conditions for asymptotic Normality of a finite-dimensional MLE will give you efficiency.

The reason is that these proofs all try to establish $$\sqrt{n}(\hat\theta-\theta_0)=-I_n^{-1}\sum_i U(X_i;\beta_0)+o_p(1)$$ where $U$ is the score and $I$ is the information, or perhaps the same thing with a more specific remainder term.

That is, the proofs work because the estimator is asymptotically equivalent to the sum of its influence functions, and the influence functions are the efficient influence functions for the parameter, whose sum has the efficient distribution.

It's quite difficult to get a consistent, asymptotically Normal MLE that isn't efficient. I don't know of any examples that would satisfy any of the standard sets of conditions for asymptotic Normality. One example is

$$X_{ij}\sim N(\mu_i,\sigma^2)$$ for $i=1,\dots,M_n$, $j=1,\dots,m_n$. The MLE of $\sigma^2$ has bias $(M_nm_n-M_n)^{-1}$. If $M_n$ is constant you get an ordinary efficient MLE. If $m_n$ is constant you get the Neyman-Scott problem and an inconsistent MLE. If you tune the rates at which $M_n$ and $m_n$ increase you can get consistency and asymptotic Normality, but a bias in the asymptotic distribution large enough to stop the estimator having the the efficient optimal MSE.