3
$\begingroup$

Can interaction be statistically significant while only one of two main effect are significant? I specifically want to know if I can report statistically significant interaction in a regression when one of the main effects hasn't reached the significance level

$\endgroup$
2
$\begingroup$

Yes.

Plot the interaction, if you can, to attempt to understand it. From the plot, and by examining the coefficient of the interaction term, you might be able to tell if its statistical significance is there simply because you have a very large sample size, which causes almost anything to be statistically significant.

If this is the case, and if your research is exploratory, you may want to delete the interaction term from the model. Deleting it might cause the non-significant main effect to become statistically significant.

Either way, an examination of the interaction plots should help you decide whether the interaction is "impressive" or not.

Additionally, you do not need to center any variables unless centering the variable helps your interpretation. Since you can get a result (without centering), then centering the variables is a waste of time. See the following on why centering is irrelevant:

$\endgroup$
  • $\begingroup$ Centering is irrelevant for the interaction but not for the coefficients of the "main effects" which are actually simple effects in this context. $\endgroup$ – David Lane May 27 '17 at 21:55
  • $\begingroup$ And what is the utility of this? "Simple effects" is the same as simple slopes? $\endgroup$ – Heteroskedastic Jim May 27 '17 at 22:16
  • $\begingroup$ The simple slope (or simple effect) at the mean value of the other variable is the average effect, and that's what you get when you center the variables. There are many situations in which the average effect is important even when there is an interaction. $\endgroup$ – David Lane May 27 '17 at 22:44
  • $\begingroup$ Well, you needn't center for this. Simple slopes analysis can easily be done without centering. For example, the pequod package in R does almost all of this for the user. $\endgroup$ – Heteroskedastic Jim May 27 '17 at 22:58
  • $\begingroup$ Good point. JMP does it implicitly too, but many popular stat packages do not. $\endgroup$ – David Lane May 28 '17 at 0:04
2
$\begingroup$

Consider the following example in R

dat = data.frame(fac = factor(rep(c(1,2), each = 10)), x = rep(seq(11,20), 2), y = c(2*seq(11,20) + 1, rev(2*seq(11,20)+1)))

#our data looks like this
png("example.png")
plot(y ~ x, data = dat, col = fac, pch = 16)
legend("top", c("1", "2"), col = 1:2, pch = 16, horiz = TRUE)
dev.off()

enter image description here

It's clear there's no relationship between 'x' and 'y' if we don't take the factor 'fac' into account, so let's model that:

model = lm(y ~ x*fac, data = dat, contrasts = list(fac = c(-1,1)))

#> coef(summary(model))
#                Estimate   Std. Error       t value      Pr(>|t|)
#(Intercept)  3.20000e+01 1.091961e-14  2.930507e+15 2.850856e-239
#x           -1.21003e-16 6.926982e-16 -1.746836e-01  8.635195e-01
#fac1         3.10000e+01 1.091961e-14  2.838928e+15 4.737922e-239
#x:fac1      -2.00000e+00 6.926982e-16 -2.887260e+15 3.616462e-239
#Warning message:
#In summary.lm(model) : essentially perfect fit: summary may be unreliable

And so we uncovered the data generating process: no mean effect of 'x' alone, but an interaction with 'fac' instead.


Bottom line is the interaction is significant (we designed it to be so!) even if only a single main effect is significant.

We could even re-parametrize our problem in a way both 'x' and 'fac' are not significant, yet 'x:fac' is!


As I hinted before, we could make all other effects non-significant simply changing offsets in the data. Here's an example:

model = lm(scale(y) ~ scale(x)*fac, data = dat, contrasts = list(fac = c(-1,1)))
# summary(model)
#
# Call:
# lm(formula = scale(y) ~ scale(x) * fac, data = dat, contrasts = list(fac = # c(-1, 1)))
# 
# Residuals:
#        Min         1Q     Median         3Q        Max 
# -5.703e-17 -2.652e-17  1.299e-18  2.777e-17  5.785e-17 
# 
# Coefficients:
#                 Estimate Std. Error    t value Pr(>|t|)    
# (Intercept)    0.000e+00  8.815e-18  0.000e+00        1    
# scale(x)       0.000e+00  9.044e-18  0.000e+00        1    
# fac1           0.000e+00  8.815e-18  0.000e+00        1    
# scale(x):fac1 -1.000e+00  9.044e-18 -1.106e+17   <2e-16 ***
$\endgroup$
1
$\begingroup$

Yes, main effects can be small or 0 while the interaction is large. Make sure in your tests of your main effects you either leave out the cross product or center the variables before multiplying. When a cross product is in the model, a "main effect" is actually a simple effect at the level of 0 for the other independent variable.

$\endgroup$
1
$\begingroup$

The short answer is: yes, you can include the significant interaction to the model even if one of the main effects is not significant. However, you should ask two questions before that:

  1. Is this interaction strong enough or it can be ignored?
  2. Does the interaction make sense, can you find a reasonable explanation for it?

The answers to those questions should help you decide what to do not just with the interaction, but all the terms under consideration.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.