There are 2 groups of people, group A and group B. The groups were NOT assigned randomly and are not the same size.

It is true that group B has a higher rate of success than group A, but this was not the result of an experiment, it was noticed during after-the-fact analysis.

People I work with are using the words "control group" and "test group" to refer to groups A and B, but to me this seems inappropriate because they were not designed prior to analysis. Do others agree with me or is it okay to blur these lines a bit?

  • $\begingroup$ This is standard terminology in en.wikipedia.org/wiki/Case-control_study, see also case-control-study. The answers below somehow fail to mention this. $\endgroup$
    – amoeba
    Jan 25, 2018 at 11:52
  • $\begingroup$ Control/test should apply to independent variables. ......... How did the A and B groups differ (how are they defined) when they were created? You seem to not have mentioned this piece of information (is it case-control or an ab-test?) . ........... E.g. If A is the current version and B refers to a change then why do you believe something is wrong with the terminology? $\endgroup$ Jan 25, 2018 at 12:09
  • 1
    $\begingroup$ @amoeba I would not be so sure whether this question is about a case-control study. The question indeed vaguely mentions something of assigning group labels 'after-the-fact'. However, it also contains a tag 'ab-test' and case-control is not really a mode of analysis in such tests. Also, qwn mentions a 'rate of success' difference between A and B rather than the dichotomous difference in 'cases' vs no cases/'control'. Since the person who asked the question has not been on the site for years, it may be unlikely to resolve this, and possibly the question should be removed or put on hold? $\endgroup$ Jan 25, 2018 at 12:24
  • $\begingroup$ @MartijnWeterings This is not my field of expertise. Feel free to downvote and to vote to close as unclear. $\endgroup$
    – amoeba
    Jan 25, 2018 at 12:29

2 Answers 2


For me, "control", "test" and "randomized" describe completely independent characteristics of the study design.

  • "control" and "treatment" describe what was done with the two groups.
  • "test" is IMHO somewhat ambiguous as it is often used for a group of cases reserved for validation purposes.
    "control" vs. "test" would suggest to me that no treatment was done but only one group received a diagnostic test. Right now, I cannot think of a situation where this would make much sense.
  • If the groups were not assigned randomly, it is not a randomized trial. Just like a trial where the treatment was not administered in a (double) blinded fashion is not a (double) blinded trial.
    This does have consequences in terms of what cannot be concluded from the trial, but it does not change the meaning of "control" group.
  • Similarly for not having the same sample size in each group: this (may) have consequences for interpretation, and a good DoE should specify in advance the sampling scheme, but whether equal sample sizes are sensible or not depends on the question at hand (e.g. it may make sense to have equal sample sizes, or it may make sense to have the sampling reflect prevalence), so we cannot say anything substantial here.

There is another use of the word control that might help you get your head around. See, statistics has had very different approaches thought time.

The test, and statistical testing approach (t-tests, wilcoxon test, anova ) are methods for analyzing trials that are expensive and made with not a lot of that at hand.

However, they are special cases of regression analysis, or non parametric regression and once you learn that you will see that the terminology sort of spills over. I learned that by reading F. Harrel's Regression Modeling Strategies.

When you perform an ANOVA you basically basically have the following model

$$ E(y_i|x_{k,i}) = x_{k,i} + e $$ with $k$ being subscript of groups, that you encode by being a dummy/binary/one-hot variable. There are some design choices you can make to have more information, and different effects measured, but it is essentially the analyst choice. But the $k$ groups can be different combinations of control and test groups, or simply a nature given such as ethnicity, gender, country of origin, time of day window.

If you are in a setting where another independent (or co-variate, since it varies together with the categorical variable $x$) you have to control your model for that new important (at least conceptually) new independent variable. So you have the following new model:

$$ E(y_i|x_k) = x_{k,i} + age_{i} + e $$

For years, this setting was/has been called ancova, but you and I know that this is just a good'ol regression model. Notice that our focus could have been the effect of age on the expected value of $y$, controlling by some categorical value $x$, and the model would still be the same.

The key take way here is. It is just naming. A control/test nomenclature means that it is a categorical variable that you will have to choose how to encode and how to analyse. However the naming schemes still have a lot of historical background (specially from clinical trials).

And that means that you should look at the model as whole, and be sure you aren't making any mistakes (such as violating any assumptions) and utilizing the context in which the data was collected into the analysis (such as censored data or measurement error). And you should as well be able to interpret the model results, regardless of how you chose to model it. Take for example a study where a high and low dosage groups are used to study a new drug. Should you encode it as two extra groups together with the control, or use the mils of the dosage as a continuous variable? It really depends on the data that you have but also in how you choose to communicate your findings (or how your field of study usually does it) all trying to optimize information usage and information gain.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.