Is Bonferroni Correction Appropriate for this Analysis?

Question

Suppose I have performed a study wherein I compare a novel intervention (N) against the standard intervention (S). Participants are assigned to N or S randomly, and the study is performed. There are two outcome variables, A and B.

My question:

With only two outcome variables, is it appropriate and/or necessary to apply a Bonferroni correction to my analysis? If N shows statistically significant improvement in both outcomes, but S does not, can I reject the null hypothesis that N has a more significant effect than S?

In other words, could the Bonferroni correction possibly lead to a false-negative when the number of comparisons is only 2? And is there a way to detect whether Bonferroni correction is leading to a false negative?

Answer 1

Strictly speaking, you should adjust for multiplicity when there is more than one comparison, so the answer is yes. Another possibility is MANOVA with a two-dimensional response.

Answer 2

If N shows statistically significant improvement in both outcomes, but S does not, can I reject the null hypothesis that N has a more significant effect than S?

"More significant effect" — yes, but is there any use in such a statement? "Significantly higher improvement" — definetely not. You have to compare two treatments explicitly to be able to claim that. Imagine treatment N making small barely significant improvement, and treatment S — small barely unsignificant one. Since effect size of S eff(S) is greater than zero, it could happen that eff(N)-eff(S) does not differ from zero siginificantly, while eff(N) itself does.