Need for bonferroni correction

Question

I need help understanding if I need to / how to apply a bonferroni correction!

I have one continuous variable and three unrelated groups that have completed scores on this variable (Groups: A,B,C). I planned to conduct 4 pairwise comparisons with this outcome variable (A-B, A-C, AB-C and A-BC). Unfortunately normality is violated so I plan to conduct non-parametric Mann Whitney U tests for these comparisons.

Do I need to apply a bonferroni correction here (.05*4?) or as I am only looking at one outcome variable does this not apply? Are there alternative methods that may be more suitable?

Answer 1

Using bonferroni correction on multiple comparisons is correct, but there are some points to consider:

• If your research is explorative or/and you don't have an adequate sample size I would avoid bonferroni correction (you already have a low power and it is really difficult you will achieve some results);

• If the comparisons are not the main endpoint of your research I would avoid bonferroni correction;

• If you will perform a lot of multiple comparisons i would use it, otherwise if you will preform few tests the correction makes little difference;

• How you will interpret the results: If the rejections of a single test will be considered a success the bonferroni correction could be a good idea otherwise if you consider a success the rejections of more than one test you could avoid it (considering the number of comparisons);

There are certainly a lot of things to consider before deciding to apply the bonferroni correction, take always in mind that it is a conservative method.

Answer 2

The need for Bonferroni correction does not follow hard and strict rules but calls for weighing up each time you consider it.

Each time you conduct a statistical test there is a risk for an alpha error and a beta error, type I and type II error. Sometimes making an beta error is bad, because if you fail to proove a drug is efficient, you might fail to hand it to a patient who should really take it. Sometimes making an alpha error is bad, because a patient suffers from the side effects of a drug, she should not have taken in the first place.

You need to consider both, the consequences of an alpha and those of an beta error and then you will have to weigh that, not in a statistical but in a human way. When you or a sensible reader of your work comes to the conclusion, that the alpha error is the big problem that needs to be controlled at the cost of a beta error, then Bonferroni correction of Bonferroni-Holm correction will have your back. However it comes at a price as it is correcting only against the overall alpha error and thus applying a Bonferroni correction may sometimes be not indicated or even unethical, even in situations where common rules may advise its use.

In a world, where p < .05 is generally considered a success of the researcher andthe researcher feels a need to defend against the accusation of obtaining that success surreptitiously it is easy to always recommend alpha-correction but we should really not live in such a world.