1
$\begingroup$

This started as a simple discussion where everyone thought they new the answer, and ended up with arguments and dozens of paper quotations. Thoughts appreciated.

We have 10 samples: Control, A, B, C, ..., I, with good enough sample size, and the groups have similar variances.

The following comparisons were made using unpaired two-sided t-tests:

A - Control

B - Control

C - Control

...

I - Control

We obtained 9 p-values. Do we have to adjust these p-values to account for family-wise issues ?

Thanks!

Improve this question
$\endgroup$
1
  • $\begingroup$ Have you noticed that all nine comparisons are correlated? (This is because they all use the same control.) $\endgroup$
    – whuber ♦
    Commented Nov 17, 2014 at 18:00

2 Answers 2

Reset to default
2
$\begingroup$

Yes, it is necessary to adjust for repeated testing to control for increasing probability of false positives. In terms of reporting, I think it is best to report both raw and adjusted p-values, specifying which correction was used (eg Bonferroni). But why did you do a series of t-tests instead of eg ANOVA with post-hoc comparisons?

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ In the same fashion, why not just code the samples as a categorical variable, and run a regression on it, which would yield straightforward p-values per parameter? You compare them to the same reference category anyway. $\endgroup$
    – Maxim.K
    Commented Nov 17, 2014 at 18:22
1
$\begingroup$

You have multiple treatment groups that are each compared to the control group, but not compared to each other. There's a method called the Dunnett procedure designed specifically for controlling the familywise error rate in this situation.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.