4
$\begingroup$

Are there any relationship between the asymptotic properties of MLEs (assuming that the regularity conditions hold)?

I mean, once I know that the MLE for $\tau(\theta)$ is asymptotically efficient and asymptotically normally distributed, how can I explain why $\tau(\hat{\theta})$ converges in probability to $\tau(\theta)$? Which results should I use to prove this convergence?

$\endgroup$
7
$\begingroup$

There is a separate proof concerning the consistency of the mle but if you take the asymptotic normality for granted, that is if you want to start from:

$$\sqrt{n} \left(\hat{\theta} - \theta \right) \xrightarrow{D} N \left(0, I(\theta)^{-1} \right)$$

(and similarly for a function of $\theta$ under the Delta method), you can just use the fact that the variance of $\hat{\theta}$ goes to zero under the regularity conditions on the Fisher Information, namely that it exists and that it is bounded. Since mean-squared error consistency implies convergence in probability, you are done.

The original proof of consistency relies on Jensen's inequality to show that the true parameter asymptotically maximizes the likelihood. Then since the mle is itself a maximizer, under some regularity conditions it can be shown that it will be arbitrarily close to the true parameter as the sample grows larger.

$\endgroup$
  • 1
    $\begingroup$ Are we assuming in some way that $\hat{\theta}$ is unbiased for $\theta$? Is it included in the assumption of normality? $\endgroup$ – PhDing Dec 30 '15 at 17:48
  • $\begingroup$ @Alessandro Yes, since the mean of the difference goes to zero and since the normal distribution is symmetric about its mean, this implies that the bias will vanish $\endgroup$ – JohnK Dec 30 '15 at 17:49
  • $\begingroup$ Oh great! Thank you John, I owe you a large percentage of my final vote! P.S. Can you link me some sources about the original proof based on Jensen's inequality? I'm curious. Thank you very much $\endgroup$ – PhDing Dec 30 '15 at 17:51
  • 1
    $\begingroup$ @Alessandro The proofs that I know come from either Wassterman's All of Statistics, chapter 9 if I remember well or even better from Introduction to Mathematical Statistics by Hogg and Craig, 7th edition, chapter 6.1. The second proof is more complete I believe. $\endgroup$ – JohnK Dec 30 '15 at 17:53
2
$\begingroup$

To generalize a bit from MLEs to general asymptotic normality results, if you know that $$ \sqrt{n}(\hat\theta_n-\theta)\to_d N(0,Avar(\hat\theta_n)) $$ it generally follows that $$ \sqrt{n}(\hat\theta_n-\theta)=\mathcal{O}_p(1) $$ and thus $$ \hat\theta_n-\theta=\mathcal{O}_p(n^{-1/2})=o_p(1) $$ so that $$ \hat\theta_n\to_p\theta $$ So if you have proven asymptotic normality, you have also proven consistency. (In the case of some proofs like exactly the MLE case, you must however take care to avoid circular reasoning, as proofs of asymptotic normality of the MLE assume consistency as a preliminary result.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for?Browse other questions tagged or ask your own question.