# Minimum sample size for unpaired t-test

Is there a "rule" to determine the minimum sample size required for a t-test to be valid?

For example, a comparison needs to be performed between the means of 2 populations.

There are 7 data points from one population and only 2 data points from the other.

Unfortunately, the experiment is very expensive and time consuming, and obtaining more data is not feasible.

Can a t-test be used? Why or why not? Please provide details (the population variances and distributions are not known).

If a t-test can not be used, can a non parametric test (Mann Whitney) be used? Why or why not?

Thanks a lot for your help! I know very little about stats, and have been searching around the internet for a detailed answer for some time.

-
This question covers similar material & will be of interest to viewers of this page: Is there a minimum sample size required for the t-test to be valid?. –  gung Oct 19 '12 at 22:34

I'd recommend using the non-parametric Mann-Whitney U test rather than an unpaired t-test here.

There's no absolute minimum sample size for the t-test, but as the sample sizes get smaller, the test becomes more sensitive to the assumption that both samples are drawn from populations with a normal distribution. With samples this small, especially with one sample of only two, you'd need to be very sure that the population distributions were normal -- and that has to be based on external knowledge, as such small samples gives very little information in themselves about the normality or otherwise of their distributions. But you say that "the population variances and distributions are not known" (my italics).

The Mann-Whitney U test does not require any assumptions about the parametric form of the distributions, requiring only the assumption that the distributions of the two groups are the same under the null hypothesis.

-

I assume you mean you have 7 data points from one group, and 2 data points from a second group, both of which are subsets of populations (e.g. subset of males and subset of females).

The maths for the t-test can be obtained from this Wikipedia page. We will assume an independent two-sample t-test, with unequal sample sizes (7 vs. 2) and unequal variances, so about half-way down that page. You can see that the calculation is based on means and standard deviations. With only 7 subjects in one group and 2 subjects in another, you cannot assume you have good estimates for either the mean or the standard deviation. For the group with 2 subjects, the mean is simply the value that lies exactly in the middle of the two data points, so it is not well estimated. For the group with 7 subjects, sample size strongly affects variances (and therefore standard deviations, which are the square root of the variance) because extreme values exert a much stronger effect when you have a smaller sample.

For example, if you look at the basic example on the Wikipedia page for standard deviation you will see that the standard deviation is 2, and the variance (square the standard deviation) is therefore 4. But if we only had the first two data points (the 9 and the 1), the variance would be 10/2 = 5 and the standard deviation would be 2.2 and if we only had the last two values (the 4 and the 16), the variance would be 20/2 = 10 and the standard deviation would be 3.2. We're still using the same values, just less of them, and we can see the effect on our estimates.

That is the problem with using inferential statistics with small sample sizes, your results will be particularly strongly affected by sampling.

Update: is there any reason why you can't simply report the results by subject and indicate that this is exploratory work? With only two cases, the data is very similar to a case study, and these are both (1) important to write up and (2) accepted practice.

-
Thanks Michelle. This is interesting and useful to know. However, what would you recommend from a practical point of view? Given this situation, what is the best way to proceed? Thanks! –  Johnny Puzzled Jan 21 '12 at 23:05
Hi Johnny Puzzled. Without more information on your exact situation I feel unable to give more guidance. –  Michelle Jan 21 '12 at 23:35
What kind of information is needed? –  Johnny Puzzled Jan 21 '12 at 23:44
Hi again, more information on your study design, such as what your data is, how you collected it, what your groups are, how the observations were selected. All I know is that you did an experiment with 9 observations (people? rats? neurons? blocks of cheese? radiation frequencies?) that are from two groups. –  Michelle Jan 21 '12 at 23:59
Let's say that the average blood flow to white matter in the brain was measured in humans using MRI. The groups are controls (7 people) and age/sex matched patients with a particular disorder (2 people). –  Johnny Puzzled Jan 22 '12 at 12:35

There is no minimum sample size for a t-test; the t-test was, in fact, designed for small samples. In the old days when tables were printed, you saw t-test tables for very small samples (as measured by df).

Of course, as with other tests, if there is a small sample only quite a large effect will be statistically significant.

-
But will the opposite also be a problem? I.e., might outliers that happened to be sampled lead to falsely rejecting the null hypothesis? Or is low power to detect differences a bigger problem? In this particular situation I am seeing a significant difference between the means but don't know how much to "trust" it. –  Johnny Puzzled Jan 21 '12 at 23:09
With n=2 you are definitely vulnerable to the influence of outliers--outliers in the population; how can a sample of 2 have an outlier within the sample? :-) I wouldn't try any inferential statistics in this situation. Prospects are poor for getting at the "truth," and you'll be leaving yourself wide open to criticism. –  rolando2 Jan 22 '12 at 0:23
The reason that the confidence interval will be wide is precisely because you might get an outlier. But t-test still assumes samples are from a normal population. –  Peter Flom Jan 22 '12 at 1:54