0
$\begingroup$

Suppose I have performed a study wherein I compare a novel intervention (N) against the standard intervention (S). Participants are assigned to N or S randomly, and the study is performed. There are two outcome variables, A and B.

My question:

With only two outcome variables, is it appropriate and/or necessary to apply a Bonferroni correction to my analysis? If N shows statistically significant improvement in both outcomes, but S does not, can I reject the null hypothesis that N has a more significant effect than S?

In other words, could the Bonferroni correction possibly lead to a false-negative when the number of comparisons is only 2? And is there a way to detect whether Bonferroni correction is leading to a false negative?

$\endgroup$
  • $\begingroup$ Do you compare N vs S for outcome A and also N vs S for outcome B? $\endgroup$ – James Oct 16 '14 at 18:25
  • $\begingroup$ Yes, although A and B are not necessarily independent. $\endgroup$ – Emily Oct 16 '14 at 18:26
0
$\begingroup$

Strictly speaking, you should adjust for multiplicity when there is more than one comparison, so the answer is yes. Another possibility is MANOVA with a two-dimensional response.

$\endgroup$
0
$\begingroup$

If N shows statistically significant improvement in both outcomes, but S does not, can I reject the null hypothesis that N has a more significant effect than S?

"More significant effect" — yes, but is there any use in such a statement? "Significantly higher improvement" — definetely not. You have to compare two treatments explicitly to be able to claim that. Imagine treatment N making small barely significant improvement, and treatment S — small barely unsignificant one. Since effect size of S eff(S) is greater than zero, it could happen that eff(N)-eff(S) does not differ from zero siginificantly, while eff(N) itself does.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.