0
$\begingroup$

Suppose I have performed a study wherein I compare a novel intervention (N) against the standard intervention (S). Participants are assigned to N or S randomly, and the study is performed. There are two outcome variables, A and B.

My question:

With only two outcome variables, is it appropriate and/or necessary to apply a Bonferroni correction to my analysis? If N shows statistically significant improvement in both outcomes, but S does not, can I reject the null hypothesis that N has a more significant effect than S?

In other words, could the Bonferroni correction possibly lead to a false-negative when the number of comparisons is only 2? And is there a way to detect whether Bonferroni correction is leading to a false negative?

$\endgroup$
2
  • $\begingroup$ Do you compare N vs S for outcome A and also N vs S for outcome B? $\endgroup$
    – James
    Oct 16, 2014 at 18:25
  • $\begingroup$ Yes, although A and B are not necessarily independent. $\endgroup$
    – Emily
    Oct 16, 2014 at 18:26

2 Answers 2

0
$\begingroup$

Strictly speaking, you should adjust for multiplicity when there is more than one comparison, so the answer is yes. Another possibility is MANOVA with a two-dimensional response.

$\endgroup$
0
$\begingroup$

If N shows statistically significant improvement in both outcomes, but S does not, can I reject the null hypothesis that N has a more significant effect than S?

"More significant effect" — yes, but is there any use in such a statement? "Significantly higher improvement" — definetely not. You have to compare two treatments explicitly to be able to claim that. Imagine treatment N making small barely significant improvement, and treatment S — small barely unsignificant one. Since effect size of S eff(S) is greater than zero, it could happen that eff(N)-eff(S) does not differ from zero siginificantly, while eff(N) itself does.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.