# Interactions make terms significant in regression when they should not be

I am writing code to prepare for running a logistic regression on real data. I have sample data for all my IVs but not for the outcome variable. There are many strong dependencies among the IVs but I have a lot of data points.

I created fake outcome data that is dependent only on a single IV main effect. The regression without interactions came out as I expected, only the single IV was significant, and the p-value was extremely low. However, when I update the formula to include all two-way interactions, the result is crazy. A large number of main effects and interactions are significant, some with fairly low p-values.

Why is this happening? And is there anything I can do about it?

Would appreciate any insight you have! Thanks!

Update 2: I have found that performing backward stepwise regression using BIC gets me down to the only factor that I made significant, so I hope that if I use that method with real data it will work out. I am still looking for some insight into what is happening here.

model2 <- step(model2.start,
direction = "backward",
k = log(nrow(model1.data)))
print(summary(model2))

Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -0.83138    0.02277 -36.515   <2e-16 ***
genderM      0.29235    0.03511   8.326   <2e-16 ***


Update: Added code and results for Peter Flom.

Formula with no interactions:

model1.start <- glm(formula = model1.formula,
data = model1.data,
print(summary(model1.start))

Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept)      -0.9597428  0.1336164  -7.183 6.83e-13 ***
userChangeCount   0.0069607  0.0257332   0.270    0.787
genderM           0.2975069  0.0356766   8.339  < 2e-16 ***
birth_year       -0.0009198  0.0019762  -0.465    0.642
statusG           0.0559423  0.0748278   0.748    0.455
statusN          -0.0646233  0.1669781  -0.387    0.699
statusS           0.0187706  0.0662441   0.283    0.777
statusU          -0.0257740  0.0832096  -0.310    0.757
collegeA          0.0129889  0.0679490   0.191    0.848
collegeB         -0.0040121  0.0788700  -0.051    0.959
collegeC         -0.1461340  0.0899802  -1.624    0.104
collegeD          0.0331471  0.0863881   0.384    0.701
collegeE          0.0453438  0.0756112   0.600    0.549
collegeF          0.0848041  0.0697141   1.216    0.224
collegeG          0.0901069  0.0849070   1.061    0.289


Using update.formula to add in interactions:

model2.formula <- update.formula(model1.formula, ~ .^2)
model2.start <- glm(formula = model2.formula,
data = model1.data,
print(summary(model2.start))

Coefficients: (8 not defined because of singularities)
Estimate Std. Error z value Pr(>|z|)
(Intercept)                      -2.2344367  0.5212149  -4.287 1.81e-05 ***
userChangeCount                   0.0212035  0.2069908   0.102 0.918410
genderM                           0.7479265  0.2957677   2.529 0.011447 *
birth_year                       -0.0412143  0.0168060  -2.452 0.014192 *
statusG                           2.6235509  0.8689383   3.019 0.002534 **
statusN                           0.9568523  0.5929653   1.614 0.106598
statusS                           0.2330824  0.5272580   0.442 0.658442
statusU                           1.1998468  0.6039849   1.987 0.046972 *
collegeA                         -1.4456800  0.8844914  -1.634 0.102159
collegeB                          0.3037282  0.3369018   0.902 0.367305
collegeC                         -0.5860054  0.7721724  -0.759 0.447909
collegeD                         -0.4193368  0.3765112  -1.114 0.265389
collegeE                         -0.0973697  0.3518518  -0.277 0.781984
collegeF                          0.5329413  0.2795513   1.906 0.056596 .
collegeG                          1.2383888  0.3673590   3.371 0.000749 ***
userChangeCount:genderM          -0.0743210  0.0550317  -1.351 0.176852
userChangeCount:birth_year       -0.0009157  0.0031425  -0.291 0.770763
userChangeCount:statusG           0.2469431  0.1298367   1.902 0.057177 .
userChangeCount:statusN          -0.5741316  0.3610974  -1.590 0.111843
userChangeCount:statusS           0.1604922  0.1174547   1.366 0.171808
userChangeCount:statusU           0.2652325  0.1428424   1.857 0.063337 .
userChangeCount:collegeA          0.0428316  0.1007660   0.425 0.670793
userChangeCount:collegeB          0.0720982  0.1183963   0.609 0.542553
userChangeCount:collegeC          0.1104534  0.1214471   0.909 0.363098
userChangeCount:collegeD         -0.1389037  0.1487849  -0.934 0.350517
userChangeCount:collegeE          0.1559318  0.1078295   1.446 0.148150
userChangeCount:collegeF          0.0575791  0.1017219   0.566 0.571364
userChangeCount:collegeG          0.0324145  0.1400930   0.231 0.817021
genderM:birth_year                0.0080877  0.0042729   1.893 0.058388 .
genderM:statusG                  -0.1330119  0.1663472  -0.800 0.423940
genderM:statusN                  -0.1387854  0.3548589  -0.391 0.695723
genderM:statusS                   0.1615500  0.1472553   1.097 0.272609
genderM:statusU                  -0.1702773  0.1870952  -0.910 0.362764
genderM:collegeA                  0.0496312  0.1399796   0.355 0.722919
genderM:collegeB                 -0.0238660  0.1620387  -0.147 0.882907
genderM:collegeC                  0.2330523  0.1905078   1.223 0.221208
genderM:collegeD                 -0.0220152  0.1874495  -0.117 0.906507
genderM:collegeE                 -0.2661413  0.1569381  -1.696 0.089917 .
genderM:collegeF                 -0.0251615  0.1491900  -0.169 0.866068
genderM:collegeG                  0.1045658  0.1781036   0.587 0.557132
birth_year:statusG                0.0007200  0.0080201   0.090 0.928468
birth_year:statusN                0.0723133  0.0308732   2.342 0.019167 *
birth_year:statusS                0.0046791  0.0055686   0.840 0.400759
birth_year:statusU               -0.0151671  0.0187630  -0.808 0.418889
birth_year:collegeA               0.0068271  0.0087997   0.776 0.437845
birth_year:collegeB               0.0203172  0.0095400   2.130 0.033197 *
birth_year:collegeC               0.0168092  0.0105057   1.600 0.109596
birth_year:collegeD              -0.0093096  0.0106617  -0.873 0.382563
birth_year:collegeE              -0.0060650  0.0076375  -0.794 0.427128
birth_year:collegeF               0.0144749  0.0073667   1.965 0.049424 *
birth_year:collegeG               0.0409526  0.0109024   3.756 0.000172 ***
statusG:collegeA                 -0.0161367  0.3181531  -0.051 0.959549
statusN:collegeA                         NA         NA      NA       NA
statusS:collegeA                  0.4597889  0.2774546   1.657 0.097486 .
statusU:collegeA                 -0.2270056  0.3392752  -0.669 0.503438
statusG:collegeB                 -0.4238954  0.3538770  -1.198 0.230971
statusN:collegeB                         NA         NA      NA       NA
statusS:collegeB                  0.4453308  0.3020272   1.474 0.140354
statusU:collegeB                 -0.4386064  0.3661169  -1.198 0.230919
statusG:collegeC                 -1.0156894  0.3707719  -2.739 0.006155 **
statusN:collegeC                         NA         NA      NA       NA
statusS:collegeC                 -0.7283356  0.3466765  -2.101 0.035649 *
statusU:collegeC                         NA         NA      NA       NA
statusG:collegeD                  0.4933988  0.3977650   1.240 0.214817
statusN:collegeD                         NA         NA      NA       NA
statusS:collegeD                  0.1421565  0.3152644   0.451 0.652053
statusU:collegeD                  0.5731627  0.4181356   1.371 0.170450
statusG:collegeE                  0.2283331  0.3918929   0.583 0.560135
statusN:collegeE                         NA         NA      NA       NA
statusS:collegeE                  0.3603496  0.3223417   1.118 0.263605
statusU:collegeE                  0.3225674  0.4006952   0.805 0.420808
statusG:collegeF                 -0.3994595  0.2919074  -1.368 0.171172
statusN:collegeF                         NA         NA      NA       NA
statusS:collegeF                  0.1203146  0.2477903   0.486 0.627286
statusU:collegeF                 -0.7253529  0.3232596  -2.244 0.024841 *
statusG:collegeG                 -1.3172171  0.3663165  -3.596 0.000323 ***
statusN:collegeG                         NA         NA      NA       NA
statusS:collegeG                 -0.1037409  0.3286267  -0.316 0.752245
statusU:collegeG                 -1.3636992  0.4203883  -3.244 0.001179 **


As you can see, now there are many significant effects, including main effects. To generate the fake data, I generated the outcome randomly and then went back and set a random 10% of males to have a positive outcome.

The coefficient for the lower order term is showing the unique contribution of that variable the prediction of the dependent variable, controlling for all of the other variables in the model. Whether you are entering interaction terms or other lower order main effects, the coefficient for the lower order term will usually change to reflect the fact that other predictors are in the model. An added issue in the model that you present is that you have a large number of predictors and interactions, some of which may attain significance by chance. A common practice in testing interaction terms involves centering the lower order variables around their mean and then computing the cross products based on the centered variables. Centering the variables is intended to reduce the linear association between the lower order terms and their cross products, but the problems you identify are likely to persist in the overall model. Ultimately, you need to test a simpler model in which the main effects and interactions are chosen more selectively. Good resources to read concerning the impact of interaction terms on main effects in regression include:

Kromrey, J.D., Foster-Johnson, L. (1998). Mean Centering in Moderated Multiple Regression: Much Ado About Nothing. Educational and Psychological Measurement, 58(1), 42-67.

Echambadi, R. (2007). Mean centering does not alleviate collinearity problems in moderated multiple regression models. Marketing Science.

Edwards, J.R. (2009). Seven deadly myths of testing moderation in organizational research. In Statistical and methodological myths and Urban Legends: Doctrine, Verity and Fable in the organizational and social sciences.

Shieh, G. (2011). Clarifying the role of mean centering in multicollinearity of interaction effects. British journal of mathematical and statistical psychology. 64(3), 462-477.

• Thanks for the list of references. I will check them out! What do you think about using stepwise selection to get down to a simpler model? Nov 20, 2012 at 4:40

Including an interaction term changes the interpretations of the lower-order coefficients. Specifically, your lower order coefficients are now interpreted as the effect of that variable at 0 on all the other variables included in the interaction term. So, if you have allowed IV1 to interact with IV2, the "main effect" of IV1 should be interpreted as the effect of IV1 at 0 on IV2. Because 0 will not necessarily be an interpretable value, this can create a spuriously "significant" main effect that does not make substantive sense.

The typical way of dealing with this issue is to mean-center all the IVs that are allowed to interact. In other words, if you're fitting the following model:

mod <- lm(dv ~ iv1 * iv2 * iv3, data = d)


Do the following before fitting the model:

d$iv1 <- d$iv1 - mean(d$iv1) d$iv2 <- d$iv2 - mean(d$iv2)
d$iv3 <- d$iv3 - mean(d\$iv3)


The simple effects of iv1, iv2, and iv3 in mod will now be the main effects of iv1, iv2, and iv3.