What is a minimum sample size for a paired t-test and what is a non-parametric equivalent if data is non-normal?

I have 4 sample pairs, let's say (x1, y1),...,(x4, y4).

• What is the minimum sample size for a paired t-test?
• Which assumption should I check for paired t-test?
• If my data is non-normal, what is an alternative non-parametric test?
• You will have very small power with only 4 pairs unless the effect size is huge (for example, if $x_i$ is the weight of a butterfly and $y_i$ is the weight of the tree you found it on). Commented Aug 16, 2011 at 23:52
• great example! I will use it in my consulting now :-) Commented Aug 16, 2011 at 23:54
• @shabby Because effect size depends on a standard deviation of a response, I don't see how butterflies and trees provides a good example.
– whuber
Commented Aug 17, 2011 at 3:36
• @whuber: good catch; I suppose comparing the logs of weights might correct for this (assuming variation and measurement noise are geometric). Commented Aug 17, 2011 at 3:41
• I'm just curious, I noticed a recent paper Jovanelly and Lane (2012) attempted to do a paired T-test with a sample size of 1 for both means. I was under the impression this was a poor idea. Can someone clarify this for me?
– user16033
Commented Oct 18, 2012 at 0:30

With such a small sample size the normality assumption is rather important. You may consider the Wilcoxon signed rank test if you think this assumption is faulty.

If the population is normally distributed, there is no minimum sample size. If the mean difference is small relative to the population variance, then you will have very little power as well. However, it is possible to get good power even with a very small sample size.

As an example suppose your pairwise differences were normally distributed with (unknown) variance $\sigma^{2} = 1$. Below are monte carlo estimates (using 10000 sims) of the power for incrementally larger values $0, .5, 1, ..., 5$ of the mean pairwise differences

      Mean Difference  Power
[1,]             0.0 0.0512
[2,]             0.5 0.1097
[3,]             1.0 0.2934
[4,]             1.5 0.5250
[5,]             2.0 0.7467
[6,]             2.5 0.8975
[7,]             3.0 0.9648
[8,]             3.5 0.9925
[9,]             4.0 0.9976
[10,]             4.5 0.9998
[11,]             5.0 0.9999


So we can see that it is possible for the paired $t$-test to still have good power when the mean difference is pretty large in comparison to the variance of the differences (at least 2x as large in this case), even if $n=4$. Please keep in mind that this all goes directly out of the window if the differences are not normally distributed.

You can look at these powers for other values of the mean difference and variance if you like using the R code below (note: the critical value for the $t$-test when $n=4$ using the usual .05 cutoff is 3.182446. The null value to be tested is assumed to be 0).

U=seq(0,5,by=.5)
V=U-U
sig=1

for(k in 1:11)
{
Z=rep(0,10000)
for(i in 1:10000)
{
diffs=rnorm(4,mean=U[k], sd=sig)
z=(mean(diffs)-0)/(sd(diffs)/sqrt(4))
Z[i]=z
}
V[k] = mean(abs(Z)>3.182446)
}
X=cbind(U,V)
colnames(X)=c("Mean Difference", "Power")
X

• In R you can get the same results using the more user friendly power.t.test, that gives you flexibility on what you keep fixed. E.g. the example above becomes for (k in 0:11){cat(sprintf("%f %f\n", k/2, power.t.test(n=4, delta=k/2, sd=1, type="paired")\$power))}
– Dr G
Commented Aug 23, 2011 at 0:51
• I read somewhere that the Wilcoxon test requires a larger sample size n > 20. Can you comment on that? Commented Apr 16, 2016 at 18:20

There is no minimum sample size for a t-test. But as @shabbychef noted, you will have very little power.

• may have very little power. If the normality assumption holds, the t-test will still be more powerful than the signed rank test, for one. I'd imagine this is also true for various other non-parametric tests. So, it still may be best to use the t-test. Commented Aug 17, 2011 at 3:12
• Technically, wouldn't the minimum sample size be two pairs, since the standard deviation/error would be undefined with only one sample? R's t.test refuses to perform the test with only one sample. Commented Oct 18, 2012 at 2:40
• @macro while the t-test does have a power advantage at the normal, the probability that the data is actually normal will be zero - and it doesn't take terribly big shifts from normality for the t-test to lose the (surprisingly small) power advantage it has at small sample sizes when its assumptions are true. It would be like saying "Why buy insurance - if I drive perfectly, it would be a waste of money" Commented Oct 18, 2012 at 3:50
• @DavidJ.Harris Yes, of course. Commented Oct 18, 2012 at 10:21
• @Glen_b, I don't understand what "the probability that the data is actually normal will be zero" is supposed to mean. If it's a statement about how all assumptions are probably wrong, then you can a poke a hole in literally any procedure (including the signed rank test - that still requires a random sample right ;)). Regarding your second point, I would say it depends very heavily on the type of departure from normality. Some departures from normality may actually increase the power of the test. Perhaps you can expand on what you meant, specifically. Commented Oct 19, 2012 at 21:20

What is the minimum sample size for a paired t-test?

Generally speaking for the ordinary paired t-test, two pairs is the smallest, yielding 1 d.f.

However, if you have some specific criterion you want to satisfy or some specific cirmcumstance you hope to deal with, that may lead to a different answer. You'd need to be more specific.

Which assumption should I check for paired t-test?

Normally*, I'd try to assess all of them, but if you only have 4 pairs, it's just about hopeless to try. You have four pair-differences, from which two d.f. would go to estimating the mean and variance of the differences (the location and scale not mattering for the assumptions), in essence leaving two d.f. to assess changing variance, dependence (in whatever form occur to you to look for, if any) and normality.

If my data is non-normal, what is an alternative non-parametric test?

Paired data: Wilcoxon signed rank test; or sign test; or any number of varieties of permutation test or bootstrap test (depending on how you construct your statistic/what exactly you want to test). All of them still have assumptions, of course.

But the t-test is at least reasonably robust to at least mild non-normality of the differences (and its the differences that are supposed to be normal). If the observations are say, mildly right-skew and not very heavy-tailed, the differences may be indistinguishable from normal even at large sample sizes. That said, there's little reason to avoid the signed rank test if non-normality is the main concern, but with 4 pairs, then you're pretty much stuck with a significance level of 12.5%

* To clarify, I mean "assess" in a fairly loose sense. I don't necessarily mean "look at my data", since the assumptions are mainly used to get approximately correct type I error rate. "Assess" will include consideration of what the likely situation would be under $$H_0$$, what kind of impact that might have on type I error rates, and so on. If I was considering power, I might also think about power under other sets of assumptions for $$H_1$$.