Testing and reporting interactions in multiple regression

Question

I have a model with two between-participants predictors -- one continuous (a), and one categorical with two levels (b) -- and two within-participants predictors, both categorical with two levels (x and z). All of my categorical predictors have been dummy-coded (i.e. contrasts set to 0 and 1).

My regression model states that there is a significant interaction between x and z:

Call:
lm(formula = y ~ a + b + x * z, data = df, contrasts = list(b = "contr.treatment", 
    x = "contr.treatment", z = "contr.treatment"))

                       Estimate Std. Error t value Pr(>|t|)    
(Intercept)              3.9392     0.1538   25.62  < 2e-16 ***
a                       -0.2155     0.0821   -2.63   0.0091 ** 
bY                      -0.3525     0.1409   -2.50   0.0129 *  
xY                       1.3770     0.1952    7.06  1.3e-11 ***
zY                       1.1740     0.1958    6.00  6.2e-09 ***
xY:zY                   -0.5754     0.2755   -2.09   0.0376 * 

A quick plot of the data appears to show that there is a larger effect of z when x is not present, but that presence of z still contributes significantly even when x is present.

My question is, how can I statistically show whether the above is true (or not)?

I've read other answers that involve changing the reference when treatment coding -- I've tried using this method, everything comes out as significant and I'm unsure how to interpret that. My intuition says I probably can't do a t-test, even though it's an interaction of purely categorical predictors.

What is the recommended method to interpret interactions arising from a multiple regression analysis?

Answer 1

@Robert Kubrick's answer is correct as far as it goes. Beyond testing for statistical significance, you will want to assess the size of any "interaction" effect.* Here, it's probably debatable whether the difference in slopes is great enough to matter in a practical sense. For your audience you will want to quantify the extent to which {the change in y for each unit change in x} differs depending on presence/absence of z. Armed with that, they'll be better informed as they make up their own mind how much of an interaction there is. The t and p statistics don't supply that information.

You wrote,

A quick plot of the data appears to show that there is a larger effect of z when x is not present, but that presence of z still contributes significantly even when x is present.

This would be true if you exchanged each instance of x for z and verse-vice-a.

---

*"You may find such effects described using the terms moderator effect, product effect, joint effect, or multiplicative effect. [...] Distinguish true interactions, which only apply in experimental studies, from the types of joint effects seen in most research, including observational, correlational, and descriptive studies. In many of these cases, the two variables which are said to "interact" are really part and parcel of rather than orthogonal to one another. Thus any investigation into joint effects is best done with careful attention to construct validity and the nature of the variables measured." From YellowBrickStats.com.

Answer 2

In your regression results the last coefficients show the partial slope of the x:z interaction, which you also illustrated in the graph. The regression output also shows the standard error of the mean for the same coefficient, and the related t-value, which indicates that the coefficient is more than 2 standard error away from 0 (2.09 exactly).

What this means is that your interaction coefficient is unlikely to be equal to 0 with more than 95% confidence level, based on CLT assumptions. In this sense it is statistically significant.

Answer 3

When examining interaction effects involving categorical predictors in a multiple regression/MANOVA context, you want to use effect coding for your categorical variables. From IDRE at UCLA:

Why use effect coding?

Here's a good question, why use effect coding instead of dummy coding? If you have several categorical variables in a model it often doesn't make much difference whether you use effect coding or dummy coding. However, if you have an interaction of two categorical variables then effect coding may provide some benefits. The primary benefit is that you get reasonable estimates of both the main effects and interaction using effect coding. With dummy coding the estimate of the interaction is fine but main effects are not "true" main effects but rather what are called simple effects, i.e., the effect of one variable at one level of the other variable. This is why most analysis of variance programs use some type of effect coding when estimating the various effects in an ANOVA model.