Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Long ago I learnt that normal distribution was necessary to use a two sample T-test. Today a colleague told me that she learnt that for N>50 normal distribution was not necessary. Is that true?

If true is that because of the central limit theorem?

share|improve this question

6 Answers 6

up vote 28 down vote accepted

Normality assumption of a t-test

Consider a large population from which you could take many different samples of a particular size. (In a particular study, you generally collect just one of these samples.)

The t-test assumes that the means of the different samples are normally distributed; it does not assume that the population is normally distributed.

By the central limit theorem, means of samples from a population with finite variance approach a normal distribution regardless of the distribution of the population. Rules of thumb say that the sample means are basically normally distributed as long as the sample size is at least 20 or 30. For a t-test to be valid on a sample of smaller size, the population distribution would have to be approximately normal.

The t-test is invalid for small samples from non-normal distributions, but it is valid for large samples from non-normal distributions.

Small samples from non-normal distributions

As Michael notes below, sample size needed for the distribution of means to approximate normality depends on the degree of non-normality of the population. For approximately normal distributions, you won't need as large sample as a very non-normal distribution.

Here are some simulations you can run in R to get a feel for this. First, here are a couple of population distributions.

curve(dnorm,xlim=c(-4,4)) #Normal
curve(dchisq(x,df=1),xlim=c(0,30)) #Chi-square with 1 degree of freedom

Next are some simulations of samples from the population distributions. In each of these lines, "10" is the sample size, "100" is the number of samples and the function after that specifies the population distribution. They produce histograms of the sample means.

hist(colMeans(sapply(rep(10,100),rnorm)),xlab='Sample mean',main='')
hist(colMeans(sapply(rep(10,100),rchisq,df=1)),xlab='Sample mean',main='')

For a t-test to be valid, these histograms should be normal.

require(car)
qqp(colMeans(sapply(rep(10,100),rnorm)),xlab='Sample mean',main='')
qqp(colMeans(sapply(rep(10,100),rchisq,df=1)),xlab='Sample mean',main='')

Utility of a t-test

I have to note that all of the knowledge I just imparted is somewhat obselete; now that we have computers, we can do better than t-tests. As Frank notes, you probably want to use Wilcoxon tests anywhere you were taught to run a t-test.

share|improve this answer
6  
Good explanation (+1). I would add, however, that the sample size needed for the distribution of means to approximate normality depnds on the degree of non-normalness of the population. For large samples there is no reason to prefer a t-test over a permutations test that makes no assumptions about the distributions. –  Michael Lew Apr 15 '11 at 2:01
1  
+1 although, as far as I know, t-test is fairly resistent to moderate deviations from normality. Also, an interesting related discussion: stats.stackexchange.com/questions/2492/… –  nico Apr 15 '11 at 7:02
4  
good answer, although there is one small detail that you missed: the distribution of the data must have finite variance. T-test is hopeless for comparing difference in location of two Cauchy distributions (or student with 2 degrees of freedom), not because it is "non-robust", but because for these distributions there is additional relevant information in the sample beyond the means and standard deviations which the t-test throws away. –  probabilityislogic Apr 24 '11 at 0:55
1  
But is there really data in reality that has has an underlying infinite variance? I would suspect that this happens only by taking ratios where the value can be arbitarily close to zero. Otherwise I disagree that there is no reason to use the t-test any more; it is so closely based on the effect size that it already remains useful for this purpose. Much more critical the Wilcoxon test tests medians and not means - in some cases this does not matter, in other it might be exactly what you want and in still other it might not be what you want at all. –  Erik Mar 7 at 10:53
1  
In addition to this, t-test also naturally yield confidence intervals for the parameter being investigated. (still upvote because of the two first paragraphs which adress the question directly, I just disagree strongly with the third) –  Erik Mar 7 at 10:58

The central limit theorem is less useful than one might think in this context. First, as someone pointed out already, one does not know if the current sample size is "large enough". Secondly, the CLT is more about achieving the desired type I error than about type II error. In other words, the t-test can be uncompetitive power-wise. That's why the Wilcoxon test is so popular. If normality holds, it is 95% as efficient as the t-test. If normality does not hold it can be arbitrarily more efficient than the t-test.

share|improve this answer
6  
(+1) Welcome to the site, which I'm glad you've found. I look forward to your participation here. –  cardinal Apr 20 '11 at 14:52
3  
(+1) Good point about the Wilcoxon. –  whuber Apr 20 '11 at 14:56

See my previous answer to a question on the robustnest off the t-test.

In particular, I recommend playing around with the onlinestatsbook applet.

The image below is based on the following scenario:

  • Null hypothesis is true
  • fairly severe skewness
  • Same distribution in both groups
  • same variance in both groups
  • sample size per group 5 (i.e., much less than 50 as per your question)
  • I pressed the 10,000 simulations button about 100 times to get up to over one million simulations.

The simulation obtained suggests that instead of getting a 5% Type I errors, I was only getting 4.5% Type I errors.

Whether you consider this robust depends on your perspective.

enter image description here

share|improve this answer
2  
+1 Good points. The power of the t-test with skewed alternatives, though, can degrade severely (to the point where it is essentially zero even for huge effect sizes). –  whuber Apr 15 '11 at 14:20

In my experience with just the one-sample t-test, I have found that the skew of the distributions is more important than the kurtosis, say. For non-skewed but fat-tailed distributions (a t with 5 degrees of freedom, a Tukey h-distribution with $h=0.24999$, etc), I have found that 40 samples has always been sufficient to get an empirical type I rate near the nominal. When the distribution is very skewed, however, you may need many many more samples.

For example, suppose you were playing the lottery. With probability $p = 10^{-4}$ you will win 100 thousand dollars, and with probability $1-p$ you will lose one dollar. If you perform a t-test for the null that the mean return is zero based on a sample of one thousand draws of this process, I don't think you are going to achieve the nominal type I rate.

edit: duh, per @whuber's catch in the comment, the example I gave did not have mean zero, so testing for mean zero has nothing to do with the type I rate.

Because the lottery example often has a sample standard deviation of zero, the t-test chokes. So instead, I give a code example using Goerg's Lambert W x Gaussian distribution. The distribution I use here has a skew of around 1355.

#hey look! I'm learning R!
library(LambertW)

Gauss_input = create_LambertW_input("normal", beta=c(0,1))
params = list(delta = c(0), gamma = c(2), alpha = 1)
LW.Gauss = create_LambertW_output(input = Gauss_input, theta = params)
#get the moments of this distribution
moms <- mLambertW(beta=c(0,1),distname=c("normal"),delta = 0,gamma = 2, alpha = 1)

test_ttest <- function(sampsize) {
    samp <- LW.Gauss$rY(params)(n=sampsize)
    tval <- t.test(samp, mu = moms$mean)
    return(tval$p.value)
}

#to replicate randomness
set.seed(1)

pvals <- replicate(1024,test_ttest(50))
#how many rejects at the 0.05 level?
print(sum(pvals < 0.05) / length(pvals))

pvals <- replicate(1024,test_ttest(250))
#how many rejects at the 0.05 level?
print(sum(pvals < 0.05) / length(pvals))

p    vals <- replicate(1024,test_ttest(1000))
#how many rejects at the 0.05 level?
print(sum(pvals < 0.05) / length(pvals))

pvals <- replicate(1024,test_ttest(2000))
#how many rejects at the 0.05 level?
print(sum(pvals < 0.05) / length(pvals))

This code gives the empirical reject rate at the nominal 0.05 level for different sample sizes. For sample of size 50, the empirical rate is 0.40 (!); for sample size 250, 0.29; for sample size 1000, 0.21; for sample size 2000, 0.18. Clearly the one-sample t-test suffers from skew.

share|improve this answer
    
In the example you're discussing the power of the test, not its size. The null, by the way, appears to be $p=0$, for which the distribution is degenerate (an atom at a single point): that's about as far from normality as one can be! –  whuber Apr 15 '11 at 14:22

The central limit theorem establishes (under the required conditions) that the numerator of the t-statistic is asymptotically normal. The t-statistic also has a denominator. To have a t-distribution you'd need the denominator to be independent and square-root-of-a-chi-square-on-its-df.

And we know it won't be independent (that characterizes the normal!)

Slutsky's theorem would give you that the t-statistic is asymptotically normal (but not necessarily at a very useful rate).

What theorem would establish that the t-statistic is approximately t-distributed when there's non-normality, and how fast it comes in?

share|improve this answer
3  
While we may not know if the sample mean and variance are independent, we always know that they are uncorrelated. This is because the sample mean is a function of bivariate sums $x_{i}+x_{j}$, and sample variance is a function of bivariate differences $x_{i}-x_{j}$ (these are called "U statistics") and we have $cov(x_{i}+x_{j},x_{i}-x_{j})=var(x_{i})-var(x_{j})+cov(x_{i},x_{j})-cov(x_{j},x‌​_{i})=0$ as long as the distribution is "homogenous" $var(x_{i})=var(x_{j})$, which is part of the problem statement. –  probabilityislogic Apr 24 '11 at 1:08
    
Unfortunately, the distinction between uncorrelated and independent is relevant if we are to be ending up with a t-distribution. –  Glen_b May 9 at 22:21

Yes, the Central Limit Theorem tells us this is true. So long as you avoid extremely heavy-tailed traits, non-Normality presents no problems in moderate-to-large samples.

Here's a helpful review paper;

http://www.annualreviews.org/doi/pdf/10.1146/annurev.publhealth.23.100901.140546

The Wilcoxon test (mentioned by others) can have terrible power when the alternative is not a location shift of the original distribution. Furthermore, the way it measures differences between distributions is not transitive.

share|improve this answer
    
Interesting points about the Wilcoxon. However, the t-test has similar difficulties: it's especially bad at detecting shifts that are accompanied by increased variance. The bit about transitivity appears to be mainly a curiosity in the present context; it's difficult to see how it's relevant to the original hypothesis test or its interpretation. (But maybe intransitivity could become important in an ANOVA or multiple comparisons setting.) –  whuber Nov 22 '11 at 20:43
    
The unequal variance t-test (which is the default in some software) doesn't have the problem with heteroskedasticity. –  guest Nov 22 '11 at 22:02
    
Regarding transitivity; reporting the sample means, or differences in means (which is natural using a t-test approach) gives the reader something they can consider when sampling from other populations. The non-transitivity of the Wilcoxon test means that this approach has no such analog; using ranks of data is a very limited approach. –  guest Nov 22 '11 at 22:13
    
(1) The Satterthwaite-Welch (unequal variance) test doesn't overcome the power loss I referred to (although it can help a little). (2) I think you're being extreme in characterizing using ranks as "limited." In his reply, @Frank Harrell was referring to studies showing how the Wilcoxon test maintains high efficiency in many settings: this demonstrates how using the ranks is both effective and more flexible, not more limited, compared to t tests. –  whuber Nov 22 '11 at 22:36
    
(1) No, but it gives the right Type I error rate, in moderate-to-large samples (2) Thanks, but I respectfully disagree. Using t-tests over Wilcoxon makes it much easier to bridge the gap between testing and using confidence intervals. If one only wants to do testing, and never looks beyond the two groups in a study, Wilcoxon of course has situations where it works well. But often we don't want to do just testing, and want to help users generalize the results to other situations; the Wilcoxon test is then not helpful. –  guest Nov 22 '11 at 23:39

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.