Does the Central Limit Theorem imply that $(\hat{X}_n - \bar{x}) = o_p(1)$ at rate $O_p(1/\sqrt{n})$?

Question

Let $\left\{\hat{X}_n\right\}$ be a sequence of estimators that converges in probability to the constant $\bar{x}$, i.e., $\left(\hat{X}_n - \bar{x}\right) = o_p(1)$.

Then say that, by some applicable version of the Central Limit Theorem (CLT), $\sqrt{n}\left(\hat{X}_n - \bar{x}\right) \overset{d}{\to} \mathcal{N}(0, v)$, where $v$ is some positive real number in which $\sqrt{n}\sqrt{\text{Var}\left[\hat{X}_n\right]} \to v$ as $n \to \infty$. Does this statement imply that $(\hat{X}_n - \bar{x}) = o_p(1)$ at rate $O_p(1/\sqrt{n})$? Intuitively, I would assume that it does since otherwise wouldn't it be the case that $\sqrt{n}\sqrt{\text{Var}\left[\hat{X}_n\right]} \to \infty$ or $\sqrt{n}\sqrt{\text{Var}\left[\hat{X}_n\right]} \to 0$ as $n \to \infty$ (if the convergence rate were slower or faster than $1/\sqrt{n}$, respectively)? Is this intuition correct? And is there a more formal argument that captures this basic logic? Would one need to prove only that $\sqrt{\text{Var}\left[\hat{X}_n\right]} \to 0$ as $n \to \infty$ at rate $1/\sqrt{n}$ and, if so, how should one formally lay out such a proof?

Thanks so much for any help you all are able to offer. Very much appreciated!

Answer 1

I think much of the confusion here arises from lack of clarity about definitions, so it is helpful to step back and define terms fully rigorously. I will then give a somewhat more general discussion than what you originally asked for in the hopes that this extra discussion gives you further opportunity to clarify and internalize the relevant definitions. First, we say that a sequence of random variables $A_n$ is $o_p(r_n)$ (also written $A_n = o_p(r_n)$) for some positive sequence $r_n$ if it is the case that for any $\varepsilon, \delta > 0$, there exists $N\in\mathbb N$ such that for all $n \geq N$,

$$\mathbb P\left(\left|\frac{A_n}{r_n}\right| \geq \delta\right) < \varepsilon$$

On the other hand, we say that this sequence of random variables is $O_p(r_n)$ (also written $A_n = O_p(r_n)$) if for any $\varepsilon > 0$, there exist $M > 0$ and $N \in \mathbb N$ such that for all $n \geq N$,

$$\mathbb P\left(\left|\frac{A_n}{r_n}\right| \geq M\right) < \varepsilon$$

In the above, we will typically refer to $r_n$ as a "rate" of convergence. Note just how similar the above definitions are. The subtle distinction is that in the definition of $o_p(r_n)$, we cannot choose $\varepsilon$ or $\delta$. Both can be arbitrary and we must fine $N$ to make the probability statement. Meanwhile, in the definition of $O_p(r_n)$, only $\varepsilon$ cannot be chosen, and $M$ can be chosen arbitrarily to make the probability statement work. This gives an immediate result

Proposition 1: $A_n = o_p(r_n) \implies A_n = O_p(r_n)$

Proof: Just take $M = \delta$ for any $\delta > 0$ in the definition of $O_p$

This proof is just formalizing the "obvious" idea that the definition of $o_p$ is more demanding than the definition of $O_p$. We can say a few more things that are relevant to your original question

Proposition 2: If $r_n > r_n'$, then (a) $A_n = o_p(r_n') \implies A_n = o_p(r_n)$, (b) $A_n = O_p(r_n') \implies A_n = O_p(r_n)$. If furthermore, $r_n/r_n' \to \infty$, then (c) $A_n = O_p(r_n') \implies A_n = o_p(r_n)$.

Proof: Note that $\left|\frac{A_n}{r_n}\right| < \left|\frac{A_n}{r_n'}\right|$, so $\mathbb P\left(\left|\frac{A_n}{r_n}\right| \geq \delta\right) \leq \mathbb P\left(\left|\frac{A_n}{r_n'}\right| \geq \delta\right)$ (this is because the probability of a subset of some event $E$ is no larger than the probability of $E$ itself). Thus, any choice of $N,M$ that works in the definitions of $O_p(r_n')$ and $o_p(r_n')$ work in the definition of $O_p(r_n)$ and $o_p(r_n)$ as well, proving (a) and (b). To prove (c), fix $\varepsilon,\delta > 0$. Then since $A_n = O_p(r_n')$, we have that for some $M > 0$, $\mathbb P(|A_n/r_n'| \geq M) \leq \varepsilon$ let $N$ be so large that for all $n \geq N$, we have $r_n/r_n' \geq M/\delta$. Then $$\mathbb P(|A_n / r_n| \geq \delta) = \mathbb P(|A_n / r_n'| \geq r_n/r_n'\delta) \leq \mathbb P(|A_n/r_n'| \geq M) > \varepsilon$$ which is the desired result.

Finally, we relate convergence in distribution to $O_p$.

Proposition 3: Suppose some sequence of random variables $\frac{X_n}{r_n}$ convergences in distribution $X$ where the distribution of $X$ satisfies $\mathbb P(X \geq x) \to 0$ as $x \to \infty$. Then $X_n = O_p(1/r_n)$

Proof: Fix $\varepsilon > 0$. By the assumption, we can choose $M$ such that $\mathbb P(|X| > M) \leq \varepsilon / 2$. By the definition of convergence in distribution, we have that there exists $N$ such that for $n \geq N$, $\mathbb |P(|X_n/r_n| \geq M) - \mathbb P(|X| \geq M)| \leq \varepsilon / 2$. But putting these two inequalities together gives $\mathbb P(|X_n / r_n| \geq M) \leq \varepsilon$, which is what we set out to prove.

So specializing this discussion to the CLT, we can take $r_n = 1/\sqrt n$. Proposition 3 tells us that $\hat X_n - \bar x = O_p(1/\sqrt n)$. Meanwhile, proposition 2(c) then further tells that $$\hat X_n - \bar x = O_p(1/\sqrt n) \implies \hat X_n - \bar x = o_p(1)$$ In fact, we can say slightly more by Proposition 2(c) and Proposition 1 that $$\hat X_n - \bar x = O_p(n^\alpha),\quad \forall 0\leq \alpha \leq \frac12$$ $$\hat X_n - \bar x = o_p(n^\alpha),\quad \forall 0\leq \alpha < \frac12$$

P.S. I typed through all of that quite quickly and need to leave my computer for a bit, so if you see something that does not make sense (like an inequality sign that goes the wrong way), there's a good chance that it is a typo. Let me know if something is still confusing and I can revise my answer later.

Answer 2

Without loss of generality assume $\bar{x} = 0$. Convergence in distribution means that $\lim F_n(x) = F(x)$ where $F_n$ is the cdf of $\sqrt{n}\hat{X}_n$ and $F$ is the cdf of the limiting normal distribution $W$.

Given $\epsilon > 0$, pick $M$ such that $P(|W| > M) < \frac{\epsilon}{4}$, and $N$ such that for all $n > N$, $|F_n(M) - F(M)| < \frac{\epsilon}{4}$ and $|F_n(-M) - F(-M)| < \frac{\epsilon}{4}$. Then we have that $P(|\sqrt{n}\hat{X}_n| < M) < \epsilon$ for all $n > N$, so $\hat{X}_n = O_p(n^{-0.5})$.