# Why can we assume a single sample is normally distributed?

As I understood it, a sampling distribution of the desired parameter is a distribution of the parameter for all possible samples with a given sample size $n$. The Central Limit Theorem states that every sampling distribution tends to a normal distribution for a large sample size $n$ even if the underlying population or the single samples are not normally distributed. The question on which $n$ is actually sufficiently large is more complicated, but a lot of sources state that $n \ge 30$ is a good rule of thumb in most cases. So far so good.

After stating the above, most authors just proceed with taking one random sample to estimate a certain population parameter and assume it is normally distributed in the calculations. For example the computation of the critical values (z-scores) is based on a normal distribution. But we just said that the underlying population might not be normally distributed and neither single samples.

How can we now assume that the single sample we took is normally distributed? Even if it has sample size $n \ge 30$, we only said $n \ge 30$ is good for the sampling distribution to be normally distributed. Why can we say it is also good for a single sample? I don't get why we can make the seemless transition from the sampling distribution to a single sample.

Here's one example to illustrate my question:

A random sample of 605 plain M&Ms contains 87 red M&Ms. Find a 95% confidence interval for the population proportion of red M&Ms.

$\hat{p} = \frac{87}{605} = 14.4%$ is a point estimate for the population proportion $p$ and the standard error is $\sigma_{\hat{p}} = \sqrt{\frac{\hat{p}\hat{q}}{n}} = 1.4\%$.

The Z-score for 95% is $Z_{\alpha / 2} = 1.96$. (<- need to assume that sample is normally distributed)

Then $E = Z_{\alpha / 2} \cdot \sigma_{\hat{p}} = 2.9\%$.

And therefore $\hat{p} − 0.029 \le p \le \hat{p} + 0.029$ or $0.115 \le p \le 0.173$ with 95% confidence.

• I think you are confusing things. The "one random sample" is in fact one random sample of your sampling distribution. So if say, n=100, then you're sampling a single instance of $(X_1+\cdots+X_n) /\sqrt{n}$, which should approximately be normal if $n$ is big enough. No claims are made on $X_i$ being approximately normal. Dec 20 '17 at 18:44
• @AlexR. I cannot really see where the random sample of the sampling distribution is taken in examples like these: "A simple random sample of 100 bottles shows that 8 of them are defective. Find a 95% confidence interval for the percentage of defective bottles manufactured by the factory.". This is just one single sample and they proceed with it like it was normally distributed. Dec 20 '17 at 18:53
• In this case, the assumption is likely that each bottle has some probability $p$ of being defective, i.e. bernoulli. Then your estimate of $p$ is: $(X_1+\cdots+X_n)/n$, where $X_i$ is the indicator for whether bottle $i$ is defective. So your "single sample" is actually a sample of 100 bottles. Dec 20 '17 at 19:01
• @AlexR Sorry, but I still don't see where the sampling distribution is used here. I included another example in the question above. Particularly the step where we use the Z-score is where one needs to assume that the sample is normally distributed, no? Dec 20 '17 at 19:28
• You're sampling bottles from the defective-bottle distribution Dec 20 '17 at 19:30

You got the sample, and use an estimator to obtain a given property, such as the mean $\hat \mu$. The value of the estimator is a random value itself, and comes from some unknown distribution, called a sampling distribution. What your ">30" rule of thumb says is that this distribution could be approximated by the normal distribution if the sample size is larger than 30 observations. I'm not here to discuss the validity of this rule itself.

So, we're not saying here that a "single sample is normally distributed." In fact I don't even understand what you mean when saying this. We're talking about the sampling distribution of the parameter estimator such as the average $\bar x=\frac 1 n \sum_{i=1}^nx_i$. We're not saying anything about the distribution of $x_i$, because the sample size does not have anything to do with it.

In your case with a proportion there's something else that is going on. The proportion comes from Binomial distribution, which can be approximated by Normal distribution when the sample size is large. I wouldn't apply your rule of thumb here, because it's obtuse in comparison to the estimator of the variance of the proportion that is based on Binomial distribution.

• Thanks, that clarified things for me. The realisation that $\hat{\mu}$ (or $\hat{p}$ in my example) comes itself from some sampling distribution for which we can apply the CLT solves my question. Dec 20 '17 at 22:46

You are talking about the sampling distribution of a function of the sample (i.e. a statistic), in this case the mean of the sample.

The magic of the central limit theorem is that, no what what distribution the sample is drawn from, it's mean is approximately normally distributed.

Think about this way: you have a population of, say, 10.000.000.000 bottles some of which are defective (however we might choose define that). Rather than counting of all them, you draw 100 samples each of 100 bottles and for each sample you compute the mean. Now, if you plot a histogram of those means you will get the sampling distribution of the mean - and it will look like a normal distribution.

We can build a small scale simmulation to show this example, here written in R:

require(magrittr)

# Setup inital parameters:
set.seed(42)
n           <- 10*10^6
draws       <- 100
sampleSize  <- 100
samples     <- vector("list", draws)

# Setup population of bottles:
tm    <- 0.3                             # True sample distribution mean
tstd  <- sqrt(0.3*(1-0.3)) / sqrt(100)   # True sample distribtion std.

population  <- rbinom(n = n, size = 1, p = tm)

# Draw samples:
samples     <- lapply(samples, sample,
# Arguments passed to the sample function:
x        = population,
size     = sampleSize,
replace  = T
)

# Calculate sample means
means <- lapply(samples, mean) %>% unlist

# Plot the sample distribution:
means %>% density %>% plot

# Overlay with normal distribution:
x <- seq(0,0.5,length = 1000)
curve(dnorm(x, mean = tm, sd = tstd), add = T, col = "blue")


This produces the following plot (where the blue line is the actual sample distribution, and the black curve is the estimated one):

Of course, this does not fit entirely perfect to the result implied by the CLT - certainly I would not call it "decidedly non-normal".

It is also worth noting some technical details, which @whuber hints to in the comments. There are in fact many central limits theorems, which implies different convergence types towards different distributions based on the different assumptions. Here I refer to the version commonly encountered in introductory stats books, which (roughly) states the sample mean follows a normal distribution under the assumption that the sample is i.id and provided the population has finite mean and variance. So long as these assumption are met, $X$ can follow any distribution. This means that sample means from a Cauchy distribution will not converge to anything in particular. Therefore the qualifier “any distribution” should be taken with a grain of salt.

Then there is the matter of how large $n$ should be before we can use the approximation, this is really case specific and so its rough to give general guidelines. I think most experienced users on this forum would say that 30 is not enough, but personally have also seen cases where 100.000 observations where considered to small a sample. For instance, if you a machine that output a bottle, you would this machine to very consistent and so a few broken bottles in small sample is likely enough to suggest that the machine needs maintenance. On the other hand, if you are dealing with some aspect of human genome you will probably need a very large sample to provide a convincing argument.

• The important assertions in this answer are not generally true. In particular, the CLT does not apply to all distributions and the distribution of the binomial mean from samples of size 100 may be decidedly non-normal.
– whuber
Dec 20 '17 at 21:56
• @whuber While you are correct, I really fail to see how it's relevant for understanding what a sampling distribution is Dec 21 '17 at 6:06