1
$\begingroup$

This started as a simple discussion where everyone thought they new the answer, and ended up with arguments and dozens of paper quotations. Thoughts appreciated.

We have 10 samples: Control, A, B, C, ..., I, with good enough sample size, and the groups have similar variances.

The following comparisons were made using unpaired two-sided t-tests:

A - Control

B - Control

C - Control

...

I - Control

We obtained 9 p-values. Do we have to adjust these p-values to account for family-wise issues ?

Thanks!

$\endgroup$
  • $\begingroup$ Have you noticed that all nine comparisons are correlated? (This is because they all use the same control.) $\endgroup$ – whuber Nov 17 '14 at 18:00
2
$\begingroup$

Yes, it is necessary to adjust for repeated testing to control for increasing probability of false positives. In terms of reporting, I think it is best to report both raw and adjusted p-values, specifying which correction was used (eg Bonferroni). But why did you do a series of t-tests instead of eg ANOVA with post-hoc comparisons?

$\endgroup$
  • $\begingroup$ In the same fashion, why not just code the samples as a categorical variable, and run a regression on it, which would yield straightforward p-values per parameter? You compare them to the same reference category anyway. $\endgroup$ – Maxim.K Nov 17 '14 at 18:22
1
$\begingroup$

You have multiple treatment groups that are each compared to the control group, but not compared to each other. There's a method called the Dunnett procedure designed specifically for controlling the familywise error rate in this situation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.