This started as a simple discussion where everyone thought they new the answer, and ended up with arguments and dozens of paper quotations. Thoughts appreciated.

We have 10 samples: Control, A, B, C, ..., I, with good enough sample size, and the groups have similar variances.

The following comparisons were made using unpaired two-sided t-tests:

A - Control

B - Control

C - Control


I - Control

We obtained 9 p-values. Do we have to adjust these p-values to account for family-wise issues ?


  • $\begingroup$ Have you noticed that all nine comparisons are correlated? (This is because they all use the same control.) $\endgroup$
    – whuber
    Nov 17, 2014 at 18:00

2 Answers 2


Yes, it is necessary to adjust for repeated testing to control for increasing probability of false positives. In terms of reporting, I think it is best to report both raw and adjusted p-values, specifying which correction was used (eg Bonferroni). But why did you do a series of t-tests instead of eg ANOVA with post-hoc comparisons?

  • 1
    $\begingroup$ In the same fashion, why not just code the samples as a categorical variable, and run a regression on it, which would yield straightforward p-values per parameter? You compare them to the same reference category anyway. $\endgroup$
    – Maxim.K
    Nov 17, 2014 at 18:22

You have multiple treatment groups that are each compared to the control group, but not compared to each other. There's a method called the Dunnett procedure designed specifically for controlling the familywise error rate in this situation.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.