21
$\begingroup$

A sequence of estimators $U_n$ for a parameter $\theta$ is asymptotically normal if $\sqrt{n}(U_n - \theta) \to N(0,v)$. (source) We then call $v$ the asymptotic variance of $U_n$. If this variance is equal to the Cramer-Rao bound, we say the estimator/sequence is asymptotically efficient.

Question: Why do we use $\sqrt{n}$ in particular?

I know that for the sample mean, $Var(\bar{X}) = \frac{\sigma^2}{n}$ and so this choice normalizes it. But since the definitions above apply to more than the sample mean, why do we still choose to normalize by $\sqrt{n}$.

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ For a good estimator, $U_n$ should have mean $\theta$, the parameter being estimated, and the variance of $U_n$ should converge to $0$, that is, the distribution of $U_n$ should be converging to a degenerate distribution with a single atom at $\theta$. But there are many different ways that this convergence can occur, e.g. $U_n \sim U(\theta-1/n,\theta+1/n)$ or $U_n\sim N(\theta,v/n)$ etc. We wish to apply the soubriquet asymptotically normal to the latter case, but not to the former case. $\endgroup$
    – Dilip Sarwate
    Jan 3, 2015 at 16:02
  • 1
    $\begingroup$ Efficient estimators are asymptotically normal. en.wikipedia.org/wiki/… $\endgroup$
    – Khashaa
    Jan 3, 2015 at 16:31
  • 1
    $\begingroup$ Might this question be better titled as "asymptotic normality" rather than "asymptotic efficiency"? It is not clear to me where "efficiency" becomes a substantive aspect of the question, rather than just the context in which "asymptotic normality" has been encountered. $\endgroup$
    – Silverfish
    Jan 5, 2015 at 15:02
  • 1
    $\begingroup$ One just has to check a proof of the asymptotic normality of MLE! The square root $n$ is to make a central limit theorem applicable to a sample average! $\endgroup$
    – Yes
    Sep 29, 2018 at 20:35

2 Answers 2

Reset to default
19
$\begingroup$

We don't get to choose here. The "normalizing" factor, in essence is a "variance-stabilizing to something finite" factor, so as for the expression not to go to zero or to infinity as sample size goes to infinity, but to maintain a distribution at the limit.

So it has to be whatever it has to be in each case. Of course it is interesting that in many cases it emerges that it has to be $\sqrt n$. (but see also @whuber's comment below).

A standard example where the normalizing factor has to be $n$, rather than $\sqrt n$ is when we have a model

$$y_t = \beta y_{t-1} + u_t, \;\; y_0 = 0,\; t=1,...,T$$

with $u_t$ white noise, and we estimate the unknown $\beta$ by Ordinary Least Squares.

If it so happens that the true value of the coefficient is $|\beta|<1$, then the the OLS estimator is consistent and converges at the usual $\sqrt n$ rate.

But if instead the true value is $\beta=1$ (i.e we have in reality a pure random walk), then the OLS estimator is consistent but will converge "faster", at rate $n$ (this is sometimes called a "superconsistent" estimator -since, I guess, so many estimators converge at rate $\sqrt n$).
In this case, to obtain its (non-normal) asymptotic distribution, we have to scale $(\hat \beta - \beta)$ by $n$ (if we scale only by $\sqrt n$ the expression will go to zero). Hamilton ch 17 has the details.

Improve this answer
$\endgroup$
9
  • 2
    $\begingroup$ Alecos, could you clarify what is being estimated in the model $y_t = y_{t-1}+u_t, u_0=0$ (where I presume you meant $y_0=0$ and the observations are subscripted $1, 2,$ etc.). Is it that in the model $y_t = \beta y_{t-1}+u_t$ the OLS estimator $\hat{\beta}$ converges at rate $\sqrt{n}$ for $|\beta| < 1$ but when $\beta=1$ convergence is at rate $n$, or is it the case that in the model $y_t = \beta y_{t-1}+u_t$ the convergence is always at rate $n$? In short, what is the significance of the statement "and $\beta=1$, i.e. a pure random walk."? $\endgroup$
    – Dilip Sarwate
    Jan 5, 2015 at 14:11
  • $\begingroup$ @DilipSarwate Thanks. Updated. I believe it is clear now. $\endgroup$
    – Alecos Papadopoulos
    Jan 5, 2015 at 14:35
  • 6
    $\begingroup$ (+1) It might be worthwhile and instructive to note that the choice of $\sqrt{n}$ (or $n$ or whatever may be appropriate) is not unique. In its stead you may use any function $f(n)$ for which the limiting value of $f(n)/\sqrt{n}$ equals unity. It is only in this broader sense that $f$ "has to be whatever it has to be." $\endgroup$
    – whuber ♦
    Jan 5, 2015 at 16:24
  • 1
    $\begingroup$ @Khashaa The OP asked about asymptotic efficiency, but in the process, it was revealed that the OP might had the wrong impression about "normalizing" factors. This is a more fundamental issue, so I chose to cover this in my answer. Nothing is said in my answer about efficiency. $\endgroup$
    – Alecos Papadopoulos
    Jan 5, 2015 at 17:55
  • 2
    $\begingroup$ Perhaps it is worth mentioning in your answer that the case with $n$ rather than $\sqrt{n}$ is called "superconsistent"? Currently the only other mention of "superconsistent" on CV which the site's search function can pick up is another one by Alecos! I think it's a good idea to make Qs and As more search-friendly. $\endgroup$
    – Silverfish
    Jan 5, 2015 at 21:36
4
$\begingroup$

You were on the right track with a sample mean variance intuition. Re-arrange the condition:

$$\sqrt{n}(U_n - \theta) \to N(0,v)$$ $$(U_n - \theta) \to \frac{N(0,v)}{\sqrt{n}}$$ $$U_n \to N(\theta,\frac{v}{n})$$

The last equation is informal. However, it's in some way more intuitive: you say that the deviation of $U_n$ from $\theta$ is becoming more like a normal distribution when $n$ increases. The variance is shrinking, but the shape becomes closer to normal distribution.

In math they don't define the convergence to the changing right hand side ($n$ is varying). That's why the same idea is expressed as the original condition, that you gave. In which the right hand side is fixed, and the left hand side converges to it.

Improve this answer
$\endgroup$
1
  • $\begingroup$ You could explain how you do the "re-arrangements". Like what properties you apply. $\endgroup$
    – mavavilj
    Dec 19, 2018 at 4:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.