# alpha value in Bonferroni corrected test

I ran some analysis with Bonferroni corrected test in R. Now my supervisor asked me what the value for alpha is for my analysis and since I haven't changed the default value it should be same as default. The problem is that I couldn't find any documentation or code that shows me the default value for alpha.

pairwise.t.test(ctData.df$SR, ctData.df$cond, p.adj = "bonf")


tl;dr; pairwise.t.test doesn't set an alpha value, it gives adjusted p-values.

pairwise.t.test returns the adjusted p-values themselves, so it's up to you to decide on your own "alpha" (confidence level cutoff), if you're going to use the so-called Neyman-Pearson approach to dichotomize results into "reject null hypothesis" vs. "fail to reject null hypothesis". Adapting the example from ?pairwise.t.test:

airquality$Month <- factor(airquality$Month, labels = month.abb[5:9])

Pairwise comparisons using t tests with pooled SD

data:  Ozone and Month

May     Jun     Jul     Aug
Jun 1.00000 -       -       -
Jul 0.00029 0.10225 -       -
Aug 0.00019 0.08312 1.00000 -
Sep 1.00000 1.00000 0.00697 0.00485

P value adjustment method: bonferroni


So you need to look at the values in the table (which are the Bonferroni-adjusted p-values) and decide on the basis of your own alpha. For example, for the comparison of August and June (adjusted p-value = 0.08312), you could decide to reject the null hypothesis if your alpha=0.1, or fail to reject it if your alpha=0.05 ...

Unsolicited PS: unless you absolutely must, there's no reason to use Bonferroni instead of the default Holm correction. ?p.adjust even says:

There seems no reason to use the unmodified Bonferroni correction because it is dominated by Holm's method, which is also valid under arbitrary assumptions.

• (+1). Just a side remark: the simple Bonferroni correction has advantages: not everyone manages to apply Holm's version correctly (e.g. a corrected p value below alpha is not necessarily "significant") and you can easily create confidence intervals corresponding to Bonferroni corrected p values. – Michael M Dec 26 '18 at 22:18
• interesting. Can you expand/give a link explaining the first point? Do you know how one would correctly interpret the values that p.adjust gives when method="holm"? – Ben Bolker Dec 26 '18 at 22:58
• It is just how Holm's correction works. So e.g. if the three p values before correction are 0.02, 0.022 and 0.04, then none of the comparisons are significant. The smallest p value is multiplied by 3 and thus not below 5%, so the procedure stops, even if 0.022 times 2 is smaller than 0.05 and 0.04 times 1 as well. – Michael M Dec 27 '18 at 8:48