What does asymptotic efficiency mean?

I read some comparison articles, and always find "asymptotic efficiency," "asymptotically less efficient," and "asymptotically normal."

I am really confused about the meaning of these words. What is the difference between "asymptotically efficient" and "asymptotic normal distribution?" I searched many website but still can not get the idea in an easy way, say with an example. Also, when I can say that the estimator is "asymptotically efficient?"

It has something to do with Cramer - Rao inequality. For most familiar distributions you know, the variance of any unbiased estimator (which coincides with mean-squared error) is at least as high as the inverse of the Fisher information, a fixed, deterministic quantity but unknown as it is a function of unknown parameters. It is the lower bound for variance of an unbiased estimator for specific parameter(s).

An estimator is efficient implies that it achieves such efficiency by providing an unbiased estimator with lowest possible variance. It also implies that it is the best unbiased estimator.

So what is "asymptotic behavior" of an estimator? It is just how "properties" of the statistics change as the sample size approaches infinity. For example, consider the biased sample variance $$S^2$$ as an estimator of a continuous distribution. The bias is $$\frac{\sigma^2}{n}$$. As $$n$$ approaches infinity, the bias approaches 0. It is an asymptotic behavior of this estimator.

Similarly, an estimator may not be the best unbiased one at a fixed sample size n, but as n approaches infinity, the variance of that estimator approaches the Cramer - Rao lower bound. That is what we call it an "asymptotically efficient" estimator.

• Thanks a lot for helping me. So, can I said that "Asymptotic efficiency" is that the estimator still provides efficient result as the sample size goes to infinity. By efficiency I mena small difference between the estimated value and the ture value. Commented Jun 8, 2022 at 10:12
• It is something else @Alice. We call it consistency. Efficiency means you do something as efficient as possible given what you have. So an unbiased estimator is efficient if and only if its variance is equal to "inverse of Fisher information at size n", the lowest theoretical variance of an unbiased estimator given that sample size. "Assymtotically efficient" means as n approaches infinity, the unbiased estimator will become an efficient one if it is not efficient when n is fixed to a particular positive integer ; or it will remain to be an efficient one if it is already efficient at any n.
– user340483
Commented Jun 8, 2022 at 11:42
• Amazing clarification. Thank you so much. Commented Jun 9, 2022 at 9:25

Asymptotic efficiency is both simpler and more complicated than finite sample efficiency. The simplest statement of it is probably the Convolution Theorem, which says that (under some assumptions, which we'll get back to) any estimator $$\hat\theta_n$$ of a parameter $$\theta$$ based on a sample of size $$n$$ can be written as $$\sqrt{n}(\hat\theta_n-\theta)\stackrel{p}{\to}Z+\Delta$$ where $$Z\sim N(0,I^{-1})$$ has variance equal to the inverse Fisher information per observation and $$\Delta$$ is pure noise independent of $$Z$$. The distribution $$Z$$ is the best possible limiting distribution of an estimator. If $$\Delta$$ is zero, so that $$\sqrt{n}\hat\theta_n-\theta)\stackrel{p}{\to}Z$$, we say $$\hat\theta_n$$ is asymptotically efficient.

The variance bound is the same as the Cramer-Rao one, so if $$\hat\theta_n$$ is unbiased and efficient in the finite-sample sense for every $$n$$ it will be asymptotically efficient. Asymptotic efficiency is a much broader concept, though: it allows for estimators to be biased as long as the limiting asymptotic distribution is unbiased. For example, maximum likelihood estimators are asymptotically efficient under fairly weak assumptions, but they are only finite-sample efficient for a few specially nice models.

There are some tricky issues with asymptotic efficiency. It's possible for an estimator to beat the Convolution Theorem at a single point or even on a dense set of measure zero. That's why the Convolution Theorem needs assumptions: it's true for regular estimators (which don't have special weird points) and it's true almost everywhere in $$\theta$$ for arbitrary estimators, and there's a stronger result called the Local Asymptotic Minimax theorem that really nails down all the loopholes. Aad van der Vaart's Asymptotic Statistics has nice coverage of these details.