3
$\begingroup$

I ran some analysis with Bonferroni corrected test in R. Now my supervisor asked me what the value for alpha is for my analysis and since I haven't changed the default value it should be same as default. The problem is that I couldn't find any documentation or code that shows me the default value for alpha.

pairwise.t.test(ctData.df$SR, ctData.df$cond, p.adj = "bonf")
$\endgroup$
4
$\begingroup$

tl;dr; pairwise.t.test doesn't set an alpha value, it gives adjusted p-values.

pairwise.t.test returns the adjusted p-values themselves, so it's up to you to decide on your own "alpha" (confidence level cutoff), if you're going to use the so-called Neyman-Pearson approach to dichotomize results into "reject null hypothesis" vs. "fail to reject null hypothesis". Adapting the example from ?pairwise.t.test:

airquality$Month <- factor(airquality$Month, labels = month.abb[5:9])
with(airquality,pairwise.t.test(Ozone,Month,p.adj="bonf"))

Pairwise comparisons using t tests with pooled SD 

data:  Ozone and Month 

    May     Jun     Jul     Aug    
Jun 1.00000 -       -       -      
Jul 0.00029 0.10225 -       -      
Aug 0.00019 0.08312 1.00000 -      
Sep 1.00000 1.00000 0.00697 0.00485

P value adjustment method: bonferroni 

So you need to look at the values in the table (which are the Bonferroni-adjusted p-values) and decide on the basis of your own alpha. For example, for the comparison of August and June (adjusted p-value = 0.08312), you could decide to reject the null hypothesis if your alpha=0.1, or fail to reject it if your alpha=0.05 ...

Unsolicited PS: unless you absolutely must, there's no reason to use Bonferroni instead of the default Holm correction. ?p.adjust even says:

There seems no reason to use the unmodified Bonferroni correction because it is dominated by Holm's method, which is also valid under arbitrary assumptions.

$\endgroup$
  • 2
    $\begingroup$ (+1). Just a side remark: the simple Bonferroni correction has advantages: not everyone manages to apply Holm's version correctly (e.g. a corrected p value below alpha is not necessarily "significant") and you can easily create confidence intervals corresponding to Bonferroni corrected p values. $\endgroup$ – Michael M Dec 26 '18 at 22:18
  • 1
    $\begingroup$ interesting. Can you expand/give a link explaining the first point? Do you know how one would correctly interpret the values that p.adjust gives when method="holm"? $\endgroup$ – Ben Bolker Dec 26 '18 at 22:58
  • $\begingroup$ It is just how Holm's correction works. So e.g. if the three p values before correction are 0.02, 0.022 and 0.04, then none of the comparisons are significant. The smallest p value is multiplied by 3 and thus not below 5%, so the procedure stops, even if 0.022 times 2 is smaller than 0.05 and 0.04 times 1 as well. $\endgroup$ – Michael M Dec 27 '18 at 8:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.