6
$\begingroup$

I can follow the proofs in which the asymptotic normal-distribution of a maximum likelihood estimator $\tilde{\theta}_n$ is derived.

however, does this already imply that the maximum likelihood estimates are asymptotically unbiased, i.e. do we have $$E(\tilde{\theta}_n) \to \theta \text{ as } n \to \infty?$$

Since I am aware that in general it is not true that convergence in distribution implies convergence in moments, an explanation would be nice.

The result is often stated e.g. in Wikipedia as "...it means that the bias of the maximum likelihood estimator is equal to zero up to the order $n^{-1/2}$")

1.) Are some kind of regularity conditions from the mle theory used to establish this result?

2.) Or is a $\sqrt{n}$-convergence of an estimator (to a normal distribution) in general already enough to establish convergence of its moments?

Note: the Wikipedia article mentions Cox, David R.; Snell, E. Joyce (1968). A general definition of residuals , as a source where the order of the bias is derived (formula (12) or (20)).

However in this paper I can't follow the arguments 100%, since their Taylor approximation of $L'(\widehat{\beta})$ is lacking the remainder term. What is the argument used here to discard it completely?

Improve this question
$\endgroup$
2
  • $\begingroup$ Heuristically, I am inclined to say yes, as the counterexamples where convergence in distribution does not imply convergence of moments usually rely on cases where some probability mass "escapes". Now, if something is (asymptotically) normal, all moments (asymptotically) exist. But a more rigorous argument would certainly be helpful here. $\endgroup$
    – Christoph Hanck
    Commented Jul 5, 2016 at 9:28
  • $\begingroup$ See also Shao, Mathematical Statistics, Section 2.5.2, whose answer to your question seems to be "yes". $\endgroup$
    – Christoph Hanck
    Commented Jul 5, 2016 at 9:39

1 Answer 1

Reset to default
3
$\begingroup$

I have presented in this answer the issues surrounding the concept of "asymptotic unbiasedness". In short, the issue is whether it is defined as "convergence of the sequence of first moments to the true value", or as "asymptotic distribution having expected value equal to the true value" (of the parameter under estimation).

Under the second approach (which is the more intuitive in my view, while the first and the one the OP discusses can be called "unbiased in the limit"), we have that asymptotic consistency of an estimator is sufficient also for asymptotic unbiasedness. Then, when the MLE is consistent (and it usually is), it will also be asymptotically unbiased.

And no, asymptotic unbiasedness as I use the term, does not guarantee "unbiasedness in the limit" (i.e. convergence of the sequence of first moments).

The conditions for the limit of the sequence of moments to equal the corresponding moment of the asymptotic distribution can be found here, and here.

Improve this answer
$\endgroup$
4
  • $\begingroup$ In fact, the second [here][stats.stackexchange.com/a/88519/28746] is where you will find all the infos. I am surprised this question does not reach that much interest though it seems a quite important subtlety. $\endgroup$
    – MysteryGuy
    Commented Apr 1 at 18:57
  • $\begingroup$ @MysteryGuy I am not sure I understand why you are pointing me towards another answer of mine... $\endgroup$
    – Alecos Papadopoulos
    Commented Apr 1 at 19:27
  • $\begingroup$ It was a general comment to encourage anyone who could be interested in diving more into it to refer to this answer specifically. Your last link, pointing to another answer of you (I know), seems really enlightening, IMO $\endgroup$
    – MysteryGuy
    Commented Apr 1 at 19:34
  • 1
    $\begingroup$ @MysteryGuy That's good practice, I just suggest, for future use, to write the intention explicitly, so that passing readers understand that the comment is not part of an internal discussion, but of broader usefulness. $\endgroup$
    – Alecos Papadopoulos
    Commented Apr 1 at 21:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.