I am running an experiment looking at brain volume changes in a rare disorder. We have a small number of patients (n = 8) but a large control group (n = 100). Some colleagues have suggested that a balanced group should be used, ie. selecting 8 control brains, but I find this counterintuitive.

Doesn't it make more sense to use the entire control group to obtain a better estimate of population brain volume?

  • $\begingroup$ See my answer in this thread: stats.stackexchange.com/questions/54693/… Basically in that framework, you'd be making p = 8/108. I don't think your colleagues are correct. Basic algebra allows you to solve that equation for the minimum detectable effect that you are powered for at your sample size. $\endgroup$ – generic_user Oct 4 '13 at 20:13

Your colleagues are incorrect.

It's possible that they're basing their suggestion on the fact that the robustness of typical parametric tests, like a t-test, to violations of assumptions is severely compromised with imbalanced N's. But if you don't violate them then a higher N is better, even if unbalanced. And you can always examine the effect of any violations through simulation.

An easy way to imagine why sampling 8 of the 100 control subjects is a bad idea, is to picture two graphs of your data that differ by the control group. Each has the patient and control brain volume measures and a confidence interval (CI) around each measure. In each graph the CI around the test group is going to be the same. But the CI around the control group will be much smaller with the full set of 100 than it will be around a sample of 8. Which graph would you like to present?

Other questions to ask yourself when deciding whether to use all 100 control samples or only 8 include: Which do you think has a better estimate of the control mean? Which would have a better estimate of the control variance? Which is more representative of the population? The answer to all of these questions is the larger group. And relating accurate estimates of your parameters is substantially more important than issues you may have with a particular test.

NOTE: It's remotely possible that the control sample of 8 CI could be smaller than the one for the full control group. However, that would likely mean that your control sample of 8 is a terrible sample and further highlights why you want to stick with the full data set.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.