# Some new questions re the Bonferroni correction

1. When the Bonferroni correction is used, what is the actual test that is conducted? For example, is it the t-test? If you know the formula, that would be great.

2. SPSS gives P values for each pairwise comparison when the Bonferroni correction option is checked. What do we compare these P values to - 0.05 (or whatever the alpha is) or 0.05/# of comparisons? In other words, is the P value given already multiplied by the # of comparisons (in which case you would compare it to alpha) but if it is not, then you would compare it to alpha/# of comparisons.

3. I find that the P values when the Bonferroni correction option is checked are consistently higher than say when compared to Tukey's HSD P values. I understand that the Bonferroni correction is more conservative but isn't that only because alpha is divided by the number of comparisons? Why is the P value itself greater?

• Bonferroni correction can be used with any collection of simultaneous hypothesis tests, it doesn't have to be a t test or any other test in particular. – dsaxton Jan 5 '16 at 18:42

Response to #1:

The Bonferroni correction can be used for any test but the typical application is to a one-way ANOVA for multiple comparisons adjustments. The formula then is a pooled-variance t-test (but this is not required and will depend upon the options you have selected and the stats package).

Response to #2:

The p-values already reflect the Bonferroni adjustment and are compared to 5% (or whatever). Bonferroni is baked into the pie, already. You can choose, when doing Bonferroni, to adjust the alpha (which is simple but somewhat rigid) or you can choose to adjust the p-values (which is more complicated but allows the bar to be at the same level (e.g., 5%). By hand, the adjusting the alpha level is simplest. By computer, let it do the work and adjust the p-value so your mind compares to a common bar.

Generally, there is an advantage in adjusting the p-values. This is because the yardstick for other adjustment methods varies depending on many factors. So ech test would require a DIFFERENT alpha level. It is simpler to adjust the p-values. So adjusting Bonferroni p-values brings it into the same framework as other methods such as Hochberg, Holms, FDR, etc.

Response to #3:

The answer here is related to #2. Tukey's is not as conservative and so it does not punish you as much for the multiple comparisons...hence p-values are lower than Bonferroni. This would also generally be true for p-values generated using methods of Hochberg, Holms, FDR, etc.

Edited Response to #3: It is probably simplest to illustrate with a couple of examples rather than feel like I'm twittering.

Let's say we have 3 p-values: 0.001, 0.010, and 0.100 and my test is at 0.05 (5%). Bonferroni says I should test at the level dividing alpha by 3. Alpha/3=0.05/3=0.017 and that is simple. But I could also adjust the p-values by multiplying them by 3: P=0.003, 0.030, and 0.300 and testing them against 0.05. Same difference, right? And it doesn't matter much does it?

But let's consider Holm's method which is a step-down procedure. Same p-values.

The p-values are already ordered so I start by testing 0.001 against 0.05/3=0.017, I test 0.010 against 0.05/2=0.025, I test 0.100 against 0.05/1=0.05. That's tedious. Simpler to adjust the p-values so that

0.001 -> 0.003
0.010 -> 0.020
0.100 -> 0.100


and test all against 0.05. That's why it is preferred to adjust the p-values rather than alpha. For basic Bonferroni, it doesn't matter. But for other procedures, it is much better to adjust the p-values.

• Thanks for the answers. But I'm still a bit fuzzy about #3. I thought the Bonferroni correction was conservative because of the modified alpha (so you're comparing it to a lower level of significance, thus making the criterion more stringent for rejecting H0). But why is the P value itself larger? – Serge Jan 5 '16 at 20:21
• If I inflate your p-value (make it bigger), it moves towards the common alpha and give the same effect as if I lowered alpha towards the fixed p-value. The effect of either change is to make the test more conservative (especially if my change in the p-value makes it >5%)! – StatNoodle Jan 6 '16 at 20:43
• Let me rephrase my question: how is the P value inflated in the Bonferroni procedure (say in an SPSS output)? Is it simply because the original P value for each pairwise comparison is multiplied by the number of possible comparisons (as in your example), which would make it then directly comparable to the original alpha? Or is there another reason for the inflated P values when the Bonferroni correction is applied (in SPSS) when compared to the P values obtained by Tukey's HSD? – Serge Jan 7 '16 at 22:41
• Exactly. It is multiplied by the number of tests. A stepdown procedure multiplies the most significant p-value by the number of tests, the next most significant by the number of tests minus one, etc. Tukey's HSD is not quite this type of test and modifies the distribution being compared to compute its p-values. – StatNoodle Jan 10 '16 at 18:08