# Bonferroni correction in one-way ANOVA for one dependent variable

I have a quick question I hope somebody could help me with; If I have two one-way ANOVA's in my research just comparing the means of three age groups for a single dependent variable (so two seperate tests both for only 1 DV), and I use Bonferroni to compare the actual differences between the age groups, do I just leave the significance level at 0.05 (I think so) or do I divide by three because of the age groups? I thought I could leave it, but I know this Bonferroni correction of the significance level exists with multiple tests so I just want to check whether that is the case here?

Kind regards :)

• If you don’t divide by the number of comparisons, you’re not doing a Bonferroni correction, so you would test at $\alpha=0.05/3$. Bonferroni, however, is conservative and quickly sucks away your power to reject. Consider another method. The p.adjust command in R has some options.
– Dave
Feb 11 '20 at 13:01
• Thanks @Dave, I see your point. If I am not dividing the significance for my three comparisons, I am not really doing a post-hoc analysis at all xd. Thanks and I will do it! Feb 11 '20 at 13:14

In the words of Jacob Cohen the question of whether, when and how to correct for multiple corrections is one where "intelligent people disagree".

One problem with the corrections is that, by lowering type 1 error they increase type 2 error.

Another is how much correction to do. What is the number of comparisons? Everything in one paper? One table? One hypothesis? Every analysis you do in your career?

Multiple testing corrections are used when testing a large number of individual hypothesis and it would not be admissible to have a large number of errors. If I am performing 10,000 tests and keep a significance level of 0.05(5%), I will obtain 500 type-1 errors (1 in every 20). The significance levels (often referred to as $$\alpha$$) is equivalent to the probability of the type-1 error (i.e., saying something is significant when it actually is not). When performing a large number of tests, we can use the Bonferroni correction (amongst many other methods) to update our significance value to $$\frac{0.05}{number of tests}$$ (e.g. 0.05/10000 = $$5\cdot 10^{-6}$$).

In conclusion, in your case (where you are performing 3 tests), you do not need to do Bonferroni correction.

• -1 if you make no correction for the three tests, at $\alpha=0.05$, your probability of a type I error is almost 15%! $1-(0.95)^3$