Minimum sample size for unpaired t-test

Question

Is there a "rule" to determine the minimum sample size required for a t-test to be valid?

For example, a comparison needs to be performed between the means of 2 populations.

There are 7 data points from one population and only 2 data points from the other.

Unfortunately, the experiment is very expensive and time consuming, and obtaining more data is not feasible.

Can a t-test be used? Why or why not? Please provide details (the population variances and distributions are not known).

If a t-test can not be used, can a non parametric test (Mann Whitney) be used? Why or why not?

Thanks a lot for your help! I know very little about stats, and have been searching around the internet for a detailed answer for some time.

Answer 1

I'd recommend using the non-parametric Mann-Whitney U test rather than an unpaired t-test here.

There's no absolute minimum sample size for the t-test, but as the sample sizes get smaller, the test becomes more sensitive to the assumption that both samples are drawn from populations with a normal distribution. With samples this small, especially with one sample of only two, you'd need to be very sure that the population distributions were normal -- and that has to be based on external knowledge, as such small samples gives very little information in themselves about the normality or otherwise of their distributions. But you say that "the population variances and distributions are not known" (my italics).

The Mann-Whitney U test does not require any assumptions about the parametric form of the distributions, requiring only the assumption that the distributions of the two groups are the same under the null hypothesis.

Answer 2

There is no minimum sample size for a t-test; the t-test was, in fact, designed for small samples. In the old days when tables were printed, you saw t-test tables for very small samples (as measured by df).

Of course, as with other tests, if there is a small sample only quite a large effect will be statistically significant.

Answer 3

I assume you mean you have 7 data points from one group, and 2 data points from a second group, both of which are subsets of populations (e.g. subset of males and subset of females).

The maths for the t-test can be obtained from this Wikipedia page. We will assume an independent two-sample t-test, with unequal sample sizes (7 vs. 2) and unequal variances, so about half-way down that page. You can see that the calculation is based on means and standard deviations. With only 7 subjects in one group and 2 subjects in another, you cannot assume you have good estimates for either the mean or the standard deviation. For the group with 2 subjects, the mean is simply the value that lies exactly in the middle of the two data points, so it is not well estimated. For the group with 7 subjects, sample size strongly affects variances (and therefore standard deviations, which are the square root of the variance) because extreme values exert a much stronger effect when you have a smaller sample.

For example, if you look at the basic example on the Wikipedia page for standard deviation you will see that the standard deviation is 2, and the variance (square the standard deviation) is therefore 4. But if we only had the first two data points (the 9 and the 1), the variance would be 10/2 = 5 and the standard deviation would be 2.2 and if we only had the last two values (the 4 and the 16), the variance would be 20/2 = 10 and the standard deviation would be 3.2. We're still using the same values, just less of them, and we can see the effect on our estimates.

That is the problem with using inferential statistics with small sample sizes, your results will be particularly strongly affected by sampling.

Update: is there any reason why you can't simply report the results by subject and indicate that this is exploratory work? With only two cases, the data is very similar to a case study, and these are both (1) important to write up and (2) accepted practice.

Answer 4

I would recommend to compare the conclusions that you get with both, the t-test and the Mann-Whitney test, and also take a look at boxplots and the profile likelihood of the mean of each population.