Using averages of two different size groups

Question

I'm not really "into" statistics, so I could use a hand on this one. I have two small groups, one has 12 students and the another one has 6.

Both groups answered a questionnaire and I'm trying to find out which of the groups did better. I'm not sure if I should be using the average of correct answers to do this comparison. Can't this be unfair with the small group? Should I be using just 6 students of the larger group?

Answer 1

Difference of means is the standard way to do this. One could also look at difference of medians. But the issue of imbalance that you worry about is really not an issue. For many distributions in practice thes means will be approximately normally distributed. The imbalance is taken into account in the use of the variance of the mean difference. To throw out data to achoeve balance is a mistake. You throw away valuable data by doing that. If the two groups have the same variability it would be best for inferenc to design the study to have equal size groups. But if results are expected to more variable in one group than the other it is actually better to take more data from the group that you expect to have the larger variability. Even when equal sample sizes is optimal practical considerations can make it in feasible. For example in a drug trial a new treatment that is expected to be effective is studied by comparing the group getting the new drug with another group that gets placebo. No one want the placebo but a controlled trial randomizing subjects to treatment or placebo is scientifically a best approach. Randomizing giving subjects an equal chance to get the placebo instead of the treatment would not encourage enrollment as muchas as if you have double or triple the chance of getting the treatment compared to the control. To increase enrollment 2:1 or 3:1 randomization is prectical better because of faster a nd large enrollment than the ideal 1:1 randomization would give and hence is used.

The two sample student t test is often used because it applies exactly to normal populations and is robust to mild departures from normality. The difficulty with your data is that the total sample size is only 18 and small or moderate differences in means may not be detected. In small samples it is even more important not to throw out data than it would be in a very large sample.