Moderation analysis with non-significant main effects but significant interaction

Question

I am doing a simple moderation analysis with one independent variable (IV), one moderator (M) and one dependent variable (DV).

Results indicate that a regression model containing both main effects (of IV and M) is non-significant. After adding the interaction term of IVxM to the previous model, the new model remains non-significant. However, the R-squared change IS significant, so the second model accounts for a significantly higher amount of the DV than model 1.

My question is:

Under these conditions, can we speak of having a moderation here?

After all, the main effects are non-significant and I believe to remember that my statistic prof told me a moderation only occurs if the model in general and both main effects are significant...

By the way, it seems as if my main effects are non-significant due to a cross-over interaction. However, I am not sure how to find out if this cross-over interaction is significant (in SPSS). Does this also have to do with the R-squared change value?

Answer 1

Unless you have a large number of subjects AND the estimate effect sizes are meaningful for your domain AND you have a solid theory that would explain why this could happen (preferably not one made up after seeing data), I would not read to much into such result.

Remember, that The Difference Between “Significant” and “Not Significant” is not Itself Statistically Significant http://www.stat.columbia.edu/~gelman/research/published/signif4.pdf

Also, as an approximation, if you $N$ subjects to estimate main effect reliably, you need roughly $16 \times N$ subjects to estimate interactions reliably.