OLS Regression: Interaction effect is significant by itself, not when included in full model--ok to report?

Question

I have a sample with 400 cases. When I run my full model, which includes 13 predictors, the interaction term is non-significant. However, when I run a model only including the three variables involved in the interaction (two main effects + interaction effect), the interaction effect is significant. I've tracked the problem down to one of the other IVs in the full model, which is extremely highly correlated with one of the items in the interaction term (r=.75), although collinearity tests indicate no problems with multicollinearity. Both the items included in the interaction term and the one which is highly correlated are composite scales, consisting of between four and nine items each.

My question is: When reporting results from my analyses, I have a hypothesis about the interaction. Is it okay to talk about the significant interaction effects that exist in the absence of controlling for this other IV, which while not theoretically important is still important to include in the regressions. If so, how do I justify the inclusion?

Answer 1

Though this is not always straightforward, I strive to pre-specify the model, making it as large as the subject matter dictates and the sample size allows. I assess interaction effects in the context of that model. As you have found, interactions are not infrequently stand-ins for collinear main effects that are omitted from the model. It is usually more sensible to make sure that the main effects are modeled "correctly", i.e., fully, before assessing interactions. I would not want to interpret an interaction as such when it is really a stand-in.

This problem can also arise when main effects are underfit by falsely assuming linearity. That false assumption can make other main effects seem to be more important than they really are (because they are somewhat collinear with omitted nonlinear terms) as well as lead to false interactions.