Some new questions re the Bonferroni correction

Question

1. When the Bonferroni correction is used, what is the actual test that is conducted? For example, is it the t-test? If you know the formula, that would be great.

2. SPSS gives P values for each pairwise comparison when the Bonferroni correction option is checked. What do we compare these P values to - 0.05 (or whatever the alpha is) or 0.05/# of comparisons? In other words, is the P value given already multiplied by the # of comparisons (in which case you would compare it to alpha) but if it is not, then you would compare it to alpha/# of comparisons.

3. I find that the P values when the Bonferroni correction option is checked are consistently higher than say when compared to Tukey's HSD P values. I understand that the Bonferroni correction is more conservative but isn't that only because alpha is divided by the number of comparisons? Why is the P value itself greater?

Answer 1

Response to #1:

The Bonferroni correction can be used for any test but the typical application is to a one-way ANOVA for multiple comparisons adjustments. The formula then is a pooled-variance t-test (but this is not required and will depend upon the options you have selected and the stats package).

Response to #2:

The p-values already reflect the Bonferroni adjustment and are compared to 5% (or whatever). Bonferroni is baked into the pie, already. You can choose, when doing Bonferroni, to adjust the alpha (which is simple but somewhat rigid) or you can choose to adjust the p-values (which is more complicated but allows the bar to be at the same level (e.g., 5%). By hand, the adjusting the alpha level is simplest. By computer, let it do the work and adjust the p-value so your mind compares to a common bar.

Generally, there is an advantage in adjusting the p-values. This is because the yardstick for other adjustment methods varies depending on many factors. So ech test would require a DIFFERENT alpha level. It is simpler to adjust the p-values. So adjusting Bonferroni p-values brings it into the same framework as other methods such as Hochberg, Holms, FDR, etc.

Response to #3:

The answer here is related to #2. Tukey's is not as conservative and so it does not punish you as much for the multiple comparisons...hence p-values are lower than Bonferroni. This would also generally be true for p-values generated using methods of Hochberg, Holms, FDR, etc.

Edited Response to #3: It is probably simplest to illustrate with a couple of examples rather than feel like I'm twittering.

Let's say we have 3 p-values: 0.001, 0.010, and 0.100 and my test is at 0.05 (5%). Bonferroni says I should test at the level dividing alpha by 3. Alpha/3=0.05/3=0.017 and that is simple. But I could also adjust the p-values by multiplying them by 3: P=0.003, 0.030, and 0.300 and testing them against 0.05. Same difference, right? And it doesn't matter much does it?

But let's consider Holm's method which is a step-down procedure. Same p-values.

The p-values are already ordered so I start by testing 0.001 against 0.05/3=0.017, I test 0.010 against 0.05/2=0.025, I test 0.100 against 0.05/1=0.05. That's tedious. Simpler to adjust the p-values so that

0.001 -> 0.003
0.010 -> 0.020
0.100 -> 0.100

and test all against 0.05. That's why it is preferred to adjust the p-values rather than alpha. For basic Bonferroni, it doesn't matter. But for other procedures, it is much better to adjust the p-values.