Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a set of data that contains cases and controls. I would like to divide the cases by a particular phenotype before running my tests, to see how these groups differ (but not necessarily to directly compare them). My feeling is that I must also randomly divide the controls between these two groups of cases before I go any further. Is this true? I've searched a bit, but I think this may be such an elementary thing that I can't find it spelled out anywhere. Would appreciate your thoughts and expertise.

share|improve this question
Why would you randomly divide the controls? If the controls vary on the same phenotype as the cases, you could divide them on that, but if they don't then you can leave them as one group. If you are using controls as matches for cases, then they will be automatically split. E.g. A comparison of "HIV+ with AIDS", "HIV+ people without AIDS" and "People without HIV" is fine. But if it was "Men with HIV" and "Women with HIV" I would want male and female control groups. – Peter Flom Jan 14 '13 at 17:24
Hi Peter, The controls do not vary on the same phenotype. I'm using "site of onset" for the cases, so some cases have onset at SiteA, and some at SiteB, and the controls obviously don't have disease onset at all. I'm not directly comparing SiteA and SiteB to each other, so I just wanted to make sure if I include all the controls in Test1: SiteA vs AllControls and then again in Test2: SiteB vs AllControls, I'm not breaking any rules regarding multiple testing or confounding. – mfk534 Jan 14 '13 at 17:33
up vote 1 down vote accepted

Based on your my understand of your description, you may legitimately "reuse" your same full control group to conduct both sets of analyses. You do not need to allot them between the two studies.

The relevant question is: Taking a single analysis in isolation, does your control group "look like" your treatment group in every way except for the factor(s) you are studying? More exactly: do your treatment group and your control group each constitute a random sample from their respective sub-populations? If, and only if, this is so may you legitimately extrapolate from observed differences in your study groups to general statements about differences between the respective sub-populations.

You will get into trouble "reusing" a control group if they were not selected randomly from the subpopulation this control group is meant to represent, and about which you hope to draw inferences.

Suppose, say, that you first conducted a study in which you compared patients with type-A illness against a second group that had previously had type-A illness but had since recovered. Suppose you then decided to study type-B illness, comparing those who have it with a control group who has never had it. You notice that none of your recovered-from-A group has ever had illness type B, so you decide to use the recovered-from-A group as your control group for your study of illness type B. In this case, re-use of a study group would be a no-no, because your control group was not selected randomly from the group of all people who are potentially at risk for getting type-B illness.

In practice, truly random sampling is hard to achieve. For example, human subjects may as a rule only be included in a study with their informed consent. If the propensity to give consent is in any way correlated with the factors you hope to study (one could imagine so in, say, certain behavioral studies), then your study will not give truly accurate results generalizable to the entire population.

Nonetheless, the test question posed above, "Do your treatment group and your control group each constitute a random sample from their respective sub-populations?" will usually steer you away from glaring errors. A careful study should always include a description of the process by which the treatment and control groups were assembled, so that the possibility of selection bias can be examined by readers and peer reviewers.

share|improve this answer
Thanks very much, Arthur. Your explanation was exactly what I needed! I appreciate it. – mfk534 Jan 15 '13 at 15:50

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.