Cental limit theorem, Chebyshev's inequality, and convergence of distributions through rescaling

Question

I've been thinking about this issue for a few days and although read some of relevant questions on this site, still couldn't get it off my mind.

Suppose we have $n$ i.i.d random variables $X_i$ with mean $\mu$ and variance $\sigma^2$. Suppose also that $Y$ is a random variable with mean zero and variance $1$ that is independent of $X_i$s, then using Chebyshev's inequality

\begin{equation} P\Big( |\frac{\sum_{i=1}^{n}X_i-n\mu}{n\sigma}-\frac{1}{\sqrt{n}}Y |\geq \epsilon\Big)\leq \frac{V[\frac{\sum_{i=1}^{n}X_i-n\mu}{n\sigma}-\frac{1}{\sqrt{n}}Y ]}{\epsilon^2}=\frac{2}{n\epsilon^2}\rightarrow 0\quad \,\, \end{equation}

for big enough $n$. Therefore for any random variable $Y(0,1)$, and scaling it with $\frac{1}{\sqrt{n}}$, meaning $\frac{Y}{\sqrt{n}}$, we can approximate the distribution of $\frac{\sum_{i=1}^{n}X_i-n\mu}{n\sigma}$.

Doesn't it contradict the CLT and unique convergence to a normal distribution? Since we can take $n=1000$ and use any other random variable. What am I missing in this interpretation?

Answer 1

The convergence to Normal in the CLT is a stronger statement than that.

1. The (weak) Law of Large Numbers says $\bar Y_n\stackrel{p}{\to} \mu$
2. Using Chebyshev's inequality you can strengthen this to get the rate of convergence $\bar Y_n-\mu=O_p(n^{-1/2})$
3. The CLT strengthens this even further to say what happens when you rescale the $O_p(n^{-1/2})$ remainder so it doesn't collapse to zero $$\sqrt{n}(\bar Y_n-\mu)\stackrel{d}{\to} N(0,\sigma^2)$$

3 implies 2 which implies 1

Answer 2

To begin with, I'm going to use some referent parts to simplify your statement. Using the sample mean $\bar{X}_n$ you can write the difference and scaled difference of interest as:

$$D_n \equiv \frac{\bar{X}_n - \mu}{\sigma / \sqrt{n}} - Y \quad \quad \quad S_n \equiv \frac{1}{\sqrt{n}} \bigg[ \frac{\bar{X}_n - \mu}{\sigma / \sqrt{n}} - Y \bigg].$$

It is simple to show that:

$$\begin{matrix} \mathbb{E}(D_n) = 0 & & & \mathbb{V}(D_n) = 2, \\[6pt] \mathbb{E}(S_n) = 0 & & & \mathbb{V}(S_n) = 2/n. \\[6pt] \end{matrix}$$

With this machinery in place, your statement uses Chebychev's rule to say that:

$$\lim_{n \rightarrow \infty} \mathbb{P}(S_n \geqslant \epsilon) \leqslant \lim_{n \rightarrow \infty} \frac{2}{n \epsilon^2} = 0.$$

Now, the limiting result you observe is merely showing that the scaled difference $S_n$ converges in probability to zero. It does not say anything about the difference quantity $D_n = \sqrt{n} S_n$ (which has a higher order than the scaled difference). Your conclusion that you can adequately approximate the distribution of $(\bar{X}_n-\mu)/\sigma$ with the distribution of $Y$ is mistaken. Your confusion here seems to arise from conflating the difference with the scaled difference. If we look at the asymptotic properties of the difference quantity $D_n$ instead of the scaled difference, we can see that this converges to the difference between a standard normal random variable and another (arbitrary) standardised random variable. That is what the CLT tells us in this case.