Central Limit Theorem Approximation and Relation to Law of Large Numbers

Assume the Linberg-Levy CLT to where we know

$$\sqrt{n}\frac{\bar{X}_n-\mu}{\sigma}\xrightarrow{d}N(0,1).$$

I feel like I commonly see then that

$$\bar{X}_n\approx N(\mu,\frac{\sigma^2}{n}),$$

but this makes no sense to me. The CLT is as $$n\rightarrow{}\infty$$, and taking that seriously, then $$\text{Var}(\bar{X}_n)\rightarrow{}0$$, and the normal converges to the delta Dirac function. This seems like a LLN result and a contradictory approximation. What is the rigorous reason on why that approximation isn't inconsistent with the CLT? Maybe a better question is how is that approximation derived, and why is it okay to use with finite $$n$$?

• I don't understand what you're trying to get at: when a sequence of random variables, such as those in the first limit, approaches a standard Normal distribution, by definition that means for sufficiently large $n$ the distributions of all the random variables at $n$ or further in the sequence are nearly standard Normal. When you scale the $n^{\text{th}}$ variable by $\sigma\sqrt{n}$ and shift it by $\mu,$ its distribution must therefore be close to Normal with a mean of $\mu$ and variance of $\sigma^2/n:$ that's a mere matter of arithmetic.
– whuber
Jan 26 '21 at 20:52
• It doesn't seem right to scale something by a variable that is being taken to infinity. Scaling by a constant I understand perfectly well, but scaling by something arbitrarily large and moving it around after already taking the limit in the proof of the CLT, confuses me. Jan 26 '21 at 20:59
• This is no different than multiplying one sequence by another. Besides, there are no infinities in evidence: everything under discussion is finite; no limit has "already been taken."
– whuber
Jan 26 '21 at 21:31

In standard formulations of analysis, saying $$\sqrt{n}\left(\frac{\bar X_n-\mu}{\sigma}\right)\stackrel{d}{\to} N(0,1)$$ is shorthand for some statement about finite $$n$$s and $$\epsilon$$s. It's not always obvious which statement, because we often don't bother with than, but one option is
"For every $$\epsilon>0$$ there exists $$N$$ such that if $$n>N$$ and $$Z\sim N(0,1)$$ and $$f$$ is a Lipschitz function bounded by $$\pm 1$$ and with Lipschitz constant bounded by 1, $$\left|E\left[ f\left(\sqrt{n}\left(\frac{\bar X_n-\mu}{\sigma}\right)\right)\right]-E\left[ f(Z)\right]\right|<\epsilon"$$
Or in slightly less detail "For every $$\epsilon>0$$ there exists $$N$$ such that if $$n>N$$, $$\sqrt{n}\left(\frac{\bar X_n-\mu}{\sigma}\right)$$ is close to $$N(0,1)$$ in the bounded Lipschitz metric"
Now, we would usually take "$$X$$ has approximately a Normal distribution" to be a scale-invariant claim. A distance in miles has approximately a Normal distribution exactly when the same distance in furlongs or nanometres has approximately a Normal distribution". In that sense, $$X_n$$ has approximately an $$N(m,s^2)$$ distribution precisely to the extent than $$(X_n-m)/s$$ has approximately a $$N(0,1)$$ distribution, which can be measured in a variety of ways, such as by the bounded Lipschitz metric.
It is also true that $$X_n$$ has approximately a Dirac delta distribution $$\delta_\mu$$. That's not a problem because it's not a contradiction. In a finistic sense, there's more than one possible approximation to the given distribution. If you insist on thinking about infinite sequences, there's more than one infinite sequence the distribution can be embedded in.