I am doing a project, where I have conducted a randomised controlled trial. I test if I can get the same results of an already published paper, but in another setting. In my created setting, I have one control group, and 5 treatment groups. The published paper got significant differences between all treatment groups and the control group separately. There is no explanation of the statistical method used in the paper.

Which statistical method is best for testing the difference between all of my treatment groups and the control group, separately? Meaning: Treatment 1 versus control, Treatment 2 versus control, etc.

Right now, I am using linear contrasts in an ANOVA framework, which only provides significant differences between one of the treatment groups and the control group (which is okay and somewhat expected).

However, I want to make sure that I can control for the effect of age, gender, and occupation. Do I use a multiple regression to do that? Does it make sense to make ANOVA’s linear contrasts to test mean differences, and then follow it up with a multiple regression?

PS: When I make the multiple regression now, I get different significance results compared to the linear contrasts in ANOVA. Does this mean anything?


Best regards,


1 Answer 1


Generally you would pre-specify your analysis before conducting the RCT to maximise credibility.

One approach, depending on your outcome (continous with reasonably normal residuals, binary, time-to-event, count data), is to do some regression (e.g. ANCOVA, logistic regression, Cox regression, negative binomial) adjusting for key covariates (ideally with the functional form pre-specified or as a factor by meaningful cuts).

Once you have non-multiplicity-adjusted p-values, for many group versus one control, there are a number of useful tests. E.g. Dunnett's test, the weighted version (if one hypothesis was more plausible than the others before seeing the data), Dunnett-Tamahane (slightly more powerful than the original test), weighted versions thereof and there have been further publications trying for some further small improvements. They all try to exploit that you know (at least for the continous data case - in the other cases you may have to estimate it from the data) the exact correlation between the test statistics.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.