# About the central limit theorem and statistical testing

Wikipedia states that

In probability theory, the central limit theorem (CLT) states that, given certain conditions, the arithmetic mean of a sufficiently large number of iterates of independent random variables, each with a well-defined expected value and well-defined variance, will be approximately normally distributed, regardless of the underlying distribution.

I am attempting to understand its implications on statistical testing.

So suppose there is some really crazy distribution. I draw samples from that distribution, and calculate the mean. By repeating this several times, I will see that the means are approximately normally distributed. Well that is cool.

However, does this mean that I may use the Student's t-test whenever I have a large sample sizes? (Since the means it compares are about normally distributed)

What if I have small sample sizes? Say two independent samples with N=10 or N=20 or something like that. Do I need to then revert to non-parametric tests?

Another thing that I don't quite understand is that suppose I know the population distribution. I draw large samples from that. The means would be again normally distributed. Should I use the t-test even if the population distribution is not normal?

Okay, so simulation time (thank you).

Lets start by looking at a case, where the central limit theorem holds. Here I compare the distribution of the sample mean from three populations each distributed as: $N (1,1), \text{exp}(\lambda = 1), \text{Weibull}(\lambda = 1, k = 1)$. I use thes specifications, since they should more or less yield the same mean, which makes a neat plot.

In R we could do something like:

# Set up sample function:
sample_mean <- function(simulations = 10000, sample_size = 1000, population){
data <- rep(0, simulations)
for (i in 1:simulations){
srs <- sample(population, sample_size, replace = TRUE)    # Simple random sample
data[i] <- mean(srs)                                      # Save the mean
}
return(data)
}

# Create empty data storage:
set.seed(42)
data <- data.frame(matrix(, nrow = 10000, ncol = 3))
colnames(data) <- c("norm_dist", "exp_dist", "wei_dist")

# Use sample function:
data$norm_dist <- sample_mean(population = rnorm(10^6, mean = 1)) data$exp_dist  <- sample_mean(population  = rexp(10^6))
data$wei_dist <- sample_mean(population = rweibull(10^6, shape = 1)) # Plot it, and compare: require(reshape) data <- melt(data) colnames(data)[2] <- "Sample_mean" ggplot(data = data, aes(x = Sample_mean, colour = variable)) + geom_density()  So the code will: Create a population of 1.000.000 numbers, from each of the above distributions. Then the function will draw a sample of 1000 (with replacement). Then rerun 10.000 times, finally it plots the distribution. This leaves us with something like this: So far so good, we get something that looks like a normal dsitribution of the sample mean - no matter the type of distribution. Now suppose that we keep a sample of 1000 in each simulation, but scale down the number of simulations to 100 then we get: Ugh! This does not look nearly as good as before. The tails are fatter, and the distribution is much less "normal"-like. This should tell you one important thing: the central limit theorem (CLT) is only an asymptotic result. Therefore to apply the CLT, for a sample of 10 or 20, would require some strong assumptions - typically that the data is normal. If you just have some "not even close to normal" data, then you can get very far off, leading to wrong conclusions. Also, the convergence in my code is quite fast. If you scale down the number of observations in each sample to say 20, then you still get pretty results. If you use "real world" data, you might need much more, to get something similar. So your question was about testing. If the distribution of the population is normal, then you you should use a t distribution since: $$t = \frac{\mu - \bar{x}}{s.e.(\bar{x})} \sim t_{n-1}$$ If the data is not normal, then we have: $$z = \frac{\mu - \bar{x}}{s.e.(\bar{x})} \sim ^{A} N(0,1)$$ But now it is only an asymptotic result, some say that this is good at 100 observations. But if the data is skewed, or is not unimodal or have many outliers, Then it is hard to justify anything for such a small sample. Futher the distinction bwtween normal or t does not matter, once when you get over 1000 observations (and 1000 is not even close to asymptotic). The p-values are essentially the same. Note that the CLT (in the usual$\bar{x}$form), requires that we have a simple random sample from a population with finite mean and varinace. This is something that many textbooks, at least in the social sciences, overlooks. This means that there actually cases where the CLT does not apply, for instance if you sample from the the Cauchy distribution. • You need more than the CLT to assert that in the limit$z = \frac{\mu - \bar{x}}{s.e.(\bar{x})} \sim ^{A} N(0,1)\$. The CLT tells you about the numerator, but you have a ratio of random variates there. (The assertion is true under certain conditions, but the CLT alone isn't sufficient.) Jul 18 '15 at 2:44