# Holm-Bonferroni correction with two interactions

I try to get a sense of the correct adjusted Holm-Bonferroni significance levels $$\alpha$$ when dealing with multiple interaction terms. To simplify, assume one has the following model: $$y=\beta_0 + \beta_1x_1 + \beta_2x_2+\beta_3x_3+\beta_4x_1x_3+\beta_5x_2x_3 + \varepsilon$$

I would publish any of the following statistical findings (Theories T):

T1a: $$\beta_1\ne0$$ and $$\beta_4=0$$

T1b: $$\beta_1\ne0$$ and $$\beta_4\ne0$$

T2a: $$\beta_2\ne0$$ and $$\beta_5=0$$

T2b: $$\beta_2\ne0$$ and $$\beta_5\ne0$$

The crucial point is that I would not publish $$\beta_1=0$$ and $$\beta_4\ne0$$ and would also not publish $$\beta_2=0$$ and $$\beta_5\ne0$$. Thus, I test $$\beta_4$$ and $$\beta_5$$ only conditional on $$\beta_1$$ and $$\beta_2$$, respectively, being significant.

Let's assume now, we have the following p-values: 0.01 for $$\beta_1$$, 0.02 for $$\beta_2$$, 0.04 for $$\beta_4$$ and 0.05 for $$\beta_5$$. We test for the overall significance level $$\alpha=0.05$$.

Using the Holm-Bonferroni approach, I would assume that the significance level for $$\beta_1$$ is $$\alpha/2=0.25$$. That is because the interaction terms $$\beta_4$$ and $$\beta_5$$ are never tested before $$\beta_1$$ and $$\beta_2$$, respectively, are significant. For $$\beta_2$$, I would put the significance level at $$\alpha/2=0.25$$. That is because I am not testing $$\beta_5$$ before $$\beta_2$$ and we already dealt with $$\beta_1$$. That means at this point, I only considered $$\beta_2$$ and $$\beta_4$$. For $$\beta_4$$, I would put the significance level at $$\alpha/2=0.25$$. That is because I already dealt with $$\beta_1$$ and $$\beta_2$$ - that is now identical to the standard Holm-Bonferroni test. Lastly for $$\beta_5$$, the significance level is $$\alpha/1=0.05$$ as usual.

Thus in this example, I would accept T1 and T3 and reject T3 and T4 using the Holm-Bonferroni correction. Is my dealing with the interaction effects $$\beta_4$$ and $$\beta_5$$ correct?

NOTE 1: In my actual problem, I have two separate models $$y=\beta_0 + \beta_1x_1 +\beta_3x_3+\beta_4x_1x_3 + \varepsilon$$ and $$y=\beta_0 + \beta_2x_2+\beta_3x_3+\beta_5x_2x_3 + \varepsilon$$ fitted with two different datasets. But I don't think this makes any difference in this case.

NOTE 2: The question How to apply Bonferroni correction when including an interaction term? has some similarities to this one, but I am not sure whether the answers also apply to my question.

• Were all these models built on the same data? – user2974951 Jan 21 at 11:37
• Model 1 and 3 use the same sample as do Model 2 and 4; i.e. there are two different datasets. – Tom Pape Jan 21 at 11:52

So you have the following situation:

Test model 1 on data 1, if significant -> additionally test model 3 on data 1
Test model 2 on data 2, if significant -> additionally test model 4 on data 2

Because these two approaches use different data sets these are two different processes.
Also, I am not sure what you mean by $$\alpha/2$$, if you are using Bonferroni then you divide by $$n$$ the number of conducted tests not the number of models.