# Bonferroni Correction - When not to use it

In a discussion with another researcher, I was informed that it is not always necessary to perform a Bonferroni correction on exploratory research if a lot of testing is required (for example, if many questions are asked and ANOVAs are required for each of them) as it will deviate further and further away from the 0.05 level.

Could anyone recommend any papers/references that explain this situation in detail to improve my understanding and for use in my dissertation?

• This is a broad question. Could you narrow it by describing the kind of "exploratory research" or procedures you plan on doing? In the meantime, because it essentially asks for a list of results, I have made it CW.
– whuber
Commented Mar 24, 2016 at 16:59

The Bonferroni correction is a pretty conservative approach to hypothesis testing. For $$n$$ tests, it requires you a p-value of $$p/n$$ where $$p$$ is your significance level. This guarantees that the probability of you getting a positive by pure chance stays below $$p$$, but sometimes it makes it go way below $$p$$, thus also losing power in your tests.
For instance, if you want to perform a million tests, a $$p$$-value of $$5\cdot10^{-8}$$ for each of them will result in an "overall $$p$$-value" of much less than $$0.05$$
This does not mean that a $$p$$-value correction is not necessary when you have a lot of tests. It's quite the opposite! If you make enough comparisons, you will eventually get false positives if you do not adjust your $$p$$-value. The issue here is that Bonferroni "takes things too far". For a more balanced approach, you may want to use something like Tukey's honest significance