5
$\begingroup$

Suppose we have two variables $x_1$ and $x_2$ and an interaction term $x_1 \cdot x_2$. Suppose we set the family-wise error rate to $\alpha = 0.05$. For the Bonferroni correction, would we look at $\alpha/2$ or $\alpha/3$?

$\endgroup$
5
  • 1
    $\begingroup$ Just out of curiosity, are you also the person behind this question: assessing-approximate-distribution-of-data-based-on-a-histogram, & this question: what-is-the-intuition-behind-conditional-gaussian-distributions? (The usernames seem similar to me.) If so, would you mind registering your account, & then merging these into 1 account? We're happy to help you, but this makes the site run smoother. $\endgroup$ Commented Oct 17, 2013 at 15:01
  • $\begingroup$ Why do you want to do that? In all of the scientific literature I know, different main and interaction effects of an ANOVA are not considered as members of the same family of comparisons. In other words, alpha is controlled for each effect by itself. Multiple comparisons are corrected for only when simple effects are tested. $\endgroup$ Commented Oct 17, 2013 at 19:47
  • 5
    $\begingroup$ That's true, @Tal, but it's simply due to tradition. That is, it's what people do because it's what people do. There is a legitimate question about whether it is the best way to go about things. Even if you believe the appropriate answer here is "no" (which is undoubtedly a defensible position), this is certainly a question worth asking. $\endgroup$ Commented Oct 17, 2013 at 20:11
  • $\begingroup$ You are doing three tests, so its $\alpha/3$. (I'd prefer the less conservative Bonferroni-Holm correction, which is almost as simple to apply). @Tal: Setting a global error in modelling would successfully avoid people doing p-value based variable selection... $\endgroup$
    – Michael M
    Commented Oct 18, 2013 at 7:55
  • 1
    $\begingroup$ @Tal, this paper actually address the issue: Cramer, A. O. J., van Ravenzwaaij, D., Matzke, D., Steingroever, H., Wetzels, R., Grasman, R. P. P. P., … Wagenmakers, E.-J. (2016). Hidden multiplicity in exploratory multiway ANOVA: Prevalence and remedies. Psychonomic Bulletin & Review, 23(2), 640–647. doi.org/10.3758/s13423-015-0913-5 $\endgroup$ Commented Jul 18, 2017 at 14:51

1 Answer 1

2
$\begingroup$

If you're testing the significance of $x_1$, $x_2$, and $x_1x_2$, then you're doing three comparisons, so if you want to use a Bonferroni (or Holm-Bonferroni) correction, it should correct for three comparisons, using $α/3$.

(As mentioned in the comments, you should use the Holm-Bonferroni rather than the Bonferroni, since the Holm-Bonferroni is uniformly more powerful and is applicable in all the same situations as the Bonferroni.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.