# Does a vast amount of probability and statistical literature make a mistake when they make use of CLT/asymptotic normality?

Suppose we toss a fair coin $$N$$ times and we are interested in the probability that we get at least $$cN$$ heads for $$c\in [0,1]$$. We can model this situation by letting $$S_N = \sum_{i=1}^N X_i$$ where $$X_i$$ are i.i.d. Bernoulli random variables with parameter $$p=1/2$$.

On page 13-14 of these notes, for the case of $$c=3/4$$, the author uses the fact that by the central limit theorem $$S_N$$ is asymptotically normally distributed to find a bound on the tails of the distribution. That is, $$Z_N = \frac{S_N - N/2}{\sqrt{N/4}} \to \mathcal{N}(0,1).$$ Thus $$P(S_N \ge \frac{3}{4}N) = P(Z_N \ge \sqrt{N/4})$$.

Let $$g \sim \mathcal{N}(0,1)$$. We have the following approximation $$P\bigg(Z_N \ge \sqrt{\frac{N}{4}}\bigg) = P\bigg(g \ge \sqrt{\frac{N}{4}}\bigg) + \varepsilon(N),$$ where the approximation error is $$\varepsilon(N) = P\bigg(Z_N \ge \sqrt{\frac{N}{4}}\bigg) - P\bigg(g \ge \sqrt{\frac{N}{4}}\bigg).$$

The author then shows that $$P(g \ge t) \le (1/\sqrt{2\pi})e^{-N/8}$$, that is, the probability decays exponentially fast with increasing $$N$$. So it looks like the probability of getting at least $$(3/4)N$$ heads decays very fast.

However, on page 14, he shows that the approximation error $$\varepsilon(N) = O(1/\sqrt{N})$$ so the approximation is no good since the error is the dominating term.

On pages 15-19, the author then goes on to

'resolve this issue, by developing alternative, direct approaches to concentration, which bypasses the central limit theorem.'

Using these alternative methods shows that $$P(S_N \ge \frac{3}{4}N)$$ does indeed decay exponentially fast.

Now, from what I understand though, the central limit theorem and asymptotic normality is used all over the place (here's one example) to find an estimate for the probability that some quantity is less than or greater than some value, or between two values. But in every one of these situations, the result will be invalid for the reason given above (the approximation error is too big and dominates the approximation).

1. So am I missing something or is a vast amount of the probability/statistical literature simply incorrect when they use the CLT and asymptotic normality to find an estimate for the probability of some event? Since they ignore the approximation error which is generally bigger than the result itself.
2. In Example 9.1.2 here, the CLT/asymptotic normality is invoked to estimate the probability that the number of heads lies behind $$40$$ and $$60$$ out of $$100$$ coin tosses and the result given is $$0.9642$$ which matches the true answer up to three decimal places. Here we see the CLT/asymptotic normality approximation has worked out very well. But why did it work so well when the approximation error is $$O(1/\sqrt{N})$$ so we should generally expect the results to be very inaccurate instead of accurate to three decimal places?

The author of the link given by the OP states near the end of p. 14 (emphasis mine)

This big error is a roadblock toward proving concentration properties for $$S_N$$ with light, exponentially decaying tails.

The author does not say that the approximation error is so big that it destroys the validity and usefulness of the approximation itself.

The author says that the approximation error in the CLT is big enough to not allow us to obtain useful bounds and rate of decays.

And the ubiquitous presence and use of the CLT is related to it functioning as an approximation, not as a tool for the latter purposes.

• If the approximation error is is so big as to not allow us to obtain useful bounds and rate of decays then that appears to automatically imply that the approximation error destroys the validity and usefulness of the approximation itself because aren't bounds and rate of decays essentially the primary purpose of the CLT and asymptotic normality? Dec 6, 2020 at 16:22
• Take confidence intervals for example. Confidence intervals are bounds, and we have seen that the CLT approximation error prevents us from obtaining accurate bounds. So why does the approximation error of the CLT and asymptotic normality not destroy the validity of confidence intervals? Dec 6, 2020 at 16:24
• @Bertus101 You have worked an example in your post where the CLT approximation worked very well. Doesn't that make you wonder? By "bounds": we mean inequalities that hold in general. The confidence interval bounds come not from such inequalitites but again, from probability estimates. Dec 6, 2020 at 16:31
• That is why I asked the question number 2. Why did the CLT happen to work so well in that case when the author of the notes I linked showed it is not valid due to the approximation error? Dec 6, 2020 at 16:55
• The procedure for obtaining confidence intervals is exactly the same as the procedure that the author used when trying to find the probability we get at least $cN$ heads from $N$ flips. In both cases, based on the CLT, the data is approximated with the normal distribution. Then then for CI's, we consider e.g. $$P\bigg(-1.96 \le \frac{\overline{X} - \mu}{\sigma/\sqrt{N}} \le 1.96\bigg),$$ whereas in the coin-tossing case the author considered $$P\bigg(\sqrt{N}/4 \le \frac{S_N - E[S_N]}{\sqrt{\text{Var}[S_N]/N}}\bigg).$$ Dec 6, 2020 at 16:55

In statistics the central limit theorem is almost always used to characterize a statistic that is a sum whose aim is to estimate its expected value (e.g., $$\bar x$$ estimating $$\mu$$). Here the approximation applies to the neighborhood of the mode of the distribution, where the error term is relatively small.

In your example the statistic is a sum but we're not asking about its behavior near the distribution mode. We're asking about tail behavior and that's where the CLT approximation will eventually be dominated by the error term.

To take this point to an extreme, consider the probability of obtaining $$S_N<0$$, or $$S_N>N$$. The CLT approximation will give positive probability where the binomial distribution obviously gives zero.

On the other hand, your second example is a question about behavior near the mode and the CLT gives a good approximation.

• This is not accurate. CLT/asymptotic approximation is used in statistical testing when we do not know the finite-sample distribution, and where the tail behavior is the issue. Dec 5, 2020 at 22:45
• You say CLT is not accurate for the tail behaviour, but the tail behaviour is what we are interested in when we form confidence intervals with the CLT. So how can confidence intervals be considered valid? Dec 6, 2020 at 16:26