Bonferroni Correction - When not to use it In a discussion with another researcher, I was informed that it is not always necessary to perform a Bonferroni correction on exploratory research if a lot of testing is required (for example, if many questions are asked and ANOVAs are required for each of them) as it will deviate further and further away from the 0.05 level.
Could anyone recommend any papers/references that explain this situation in detail to improve my understanding and for use in my dissertation?
Many thanks in advance!
 A: You should generally address the issue of multiple testing in some way. That doesn't mean Bonferroni is the best approach in all cases, however. Different methods address different error rates and the proper method depends on the goals of the testing and the consequences of making a Type I error. Try this paper:
Frane, A. V. (2015). Planned Hypothesis Tests Are Not Necessarily Exempt From Multiplicity Adjustment. Journal of Research Practice, 11(1). Available from http://jrp.icaap.org/index.php/jrp/article/view/514/417
A: The Bonferroni correction is a pretty conservative approach to hypothesis testing. For $n$ tests, it requires you a p-value of $p/n$ where $p$ is your significance level. This guarantees that the probability of you getting a positive by pure chance stays below $p$, but sometimes it makes it go way below $p$, thus also losing power in your tests.
For instance, if you want to perform a million tests, a $p$-value of $5\cdot10^{-8}$ for each of them will result in an "overall $p$-value" of much less than $0.05$
This does not mean that a $p$-value correction is not necessary when you have a lot of tests. It's quite the opposite! If you make enough comparisons, you will eventually get false positives if you do not adjust your $p$-value. The issue here is that Bonferroni "takes things too far". For a more balanced approach, you may want to use something like Tukey's honest significance
