5
$\begingroup$

I can follow the proofs in which the asymptotic normal-distribution of a maximum likelihood estimator $\tilde{\theta}_n$ is derived.

however, does this already imply that the maximum likelihood estimates are asymptotically unbiased, i.e. do we have $$E(\tilde{\theta}_n) \to \theta \text{ as } n \to \infty?$$

Since I am aware that in general it is not true that convergence in distribution implies convergence in moments, an explanation would be nice.

The result is often stated e.g. in Wikipedia as "...it means that the bias of the maximum likelihood estimator is equal to zero up to the order $n^{-1/2}$")

1.) Are some kind of regularity conditions from the mle theory used to establish this result?

2.) Or is a $\sqrt{n}$-convergence of an estimator (to a normal distribution) in general already enough to establish convergence of its moments?

Note: the Wikipedia article mentions Cox, David R.; Snell, E. Joyce (1968). A general definition of residuals , as a source where the order of the bias is derived (formula (12) or (20)).

However in this paper I can't follow the arguments 100%, since their Taylor approximation of $L'(\widehat{\beta})$ is lacking the remainder term. What is the argument used here to discard it completely?

$\endgroup$
  • $\begingroup$ Heuristically, I am inclined to say yes, as the counterexamples where convergence in distribution does not imply convergence of moments usually rely on cases where some probability mass "escapes". Now, if something is (asymptotically) normal, all moments (asymptotically) exist. But a more rigorous argument would certainly be helpful here. $\endgroup$ – Christoph Hanck Jul 5 '16 at 9:28
  • $\begingroup$ See also Shao, Mathematical Statistics, Section 2.5.2, whose answer to your question seems to be "yes". $\endgroup$ – Christoph Hanck Jul 5 '16 at 9:39
2
$\begingroup$

I have presented in this answer the issues surrounding the concept of "asymptotic unbiasedness". In short, the issue is whether it is defined as "convergence of the sequence of first moments to the true value", or as "asymptotic distribution having expected value equal to the true value" (of the parameter under estimation).

Under the second approach (which is the more intuitive in my view, while the first and the one the OP discusses can be called "unbiased in the limit"), we have that asymptotic consistency of an estimator is sufficient also for asymptotic unbiasedness. Then, when the MLE is consistent (and it usually is), it will also be asymptotically unbiased.

And no, asymptotic unbiasedness as I use the term, does not guarantee "unbiasedness in the limit" (i.e. convergence of the sequence of first moments).

The conditions for the limit of the sequence of moments to equal the corresponding moment of the asymptotic distribution can be found here, and here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.