Adding a 2nd interaction term makes 1st interaction term and the 2nd interaction term insignificant

Question

I'm running a multiple linear OLS regression (X => Y) on a sample with 125 cases. My regression has two moderator variables (Z and W). Z and W correlate at about .4 but Z and W are two distinct/discriminant concepts (empirically tested - multicolinearity is not present (VIFs are low)).

When including each interaction term individually (XZ alone without XW and vice versa) the interaction terms are significant. However, when I include both interaction terms (XZ and XW) in a full model both become insignificant.

Theoretically speaking, both interaction terms change the relationship between X and Y similarly but through different mechanisms. That is, both interaction terms increase employee satisfaction which strengthens the relationship between X and Y (but Z and W do so through different mechanisms since Z and W are distinct concepts).

How would you explain the insignificant interaction terms in the full model? Is this some kind of suppression effect?

Answer 1

I have been struggling with how best to answer this question specific to the context provided, and I'm not sure if my attempt is going to clarify what is happening...or just make things more confusing.

In brief, this is probably an example of a confounding effect. However, the tricky part is that it is confounding via the interaction (e.g., difference in slopes), and I'm not sure you have distinct groups in this context.

That said, here's a context we can use: (1a) we will predict lung capacity from height and gender. If we start by predicting lung capacity from gender, we would find a statistically significant difference. However, if we predict lung capacity from both gender and height, we probably will still see a significant effect by gender, but the gender coefficients between the bivariate model (gender only) and the multiple regression model (gender and height) are probably going to be different. This is because height would be a confounding variable...the two gender groups will have different average heights, and consequently, when we include both in the model, the gender effect in the multiple regression (MR) model is reporting the effect AFTER having accounted for the effect from height.

Now, let me move to a slightly different scenario that is closer to what is probably happening in your data: (1b) this time we will predict lung capacity from treatment group and height. It is possible to run a set of models in which treatment is significant in the bivariate model (treatment effect only), but it is not significant in the MR model (treatment and height effects). Now, the issue is the same as in the example above...but the effect may not be as pronounced. You might think, well, let's just run a t-test to see if the heights are different between the treatment groups. It turns out, that even if there is a difference that doesn't reach significance, it is still possible for that "small" difference to result in a confounding effect. That is, the small difference in heights among the treatment groups may be enough to cause the significance of the treatment predictor to change from the bivariate model to the MR model.

Now, two issues make it tricky to extrapolate to your context. The first is that I was looking at a confounding variable affecting the effect of a grouping variable. It sounds as though all your variables are scalar (not categorical). Well, in this case, you can imagine splitting your data set into the low and high X group. This would allow us to transfer the concept from a categorical variable to a scalar variable. The much trickier part is that you are actually looking at an interaction effect (which is interpreted as the change of slope for the different groups...or in your case, for the differing levels of the X variable).

In your context, accounting for the difference of slopes with only one interaction is coming up significant. However, this affect is confounded by the other interaction term. That is, when both interaction terms are included, the effect appears to disappear.

I'll stop here, and provide any follow-up as requested.