Should I Include the predictor variables alongside the interaction coefficient within a linear regression?

Question

I'm currently using R to run a linear regression with 3 predictors. I am analysing the outcome likelihood of electoral success by descriptive identity of election candidates in Germany.

My aim is to find the interaction between female, electoral tier and political party as well as between female and electoral tier. I see three possible ways to do this.

Option 1 (estimating both interaction terms and predictor variables in one model)

lm(formula = elected ~ female + estier + PartyID + female:estier + female:estier:PartyID)
                           Estimate Std. Error t value Pr(>|t|)    
(Intercept)                 0.18082    0.02501   7.230 6.40e-13 ***
female                     -0.02850    0.02374  -1.201  0.23000    
estier                      0.02393    0.02189   1.093  0.27456    
PartyIDCDU                  0.07542    0.02704   2.789  0.00533 ** 
PartyIDGREEN                0.02731    0.02867   0.953  0.34088    
PartyIDSPD                  0.16666    0.02871   5.804 7.29e-09 ***
female:estier              -0.12361    0.07676  -1.610  0.10742    
female:estier:PartyIDCDU    0.24182    0.08886   2.721  0.00655 ** 
female:estier:PartyIDGREEN -0.01744    0.08233  -0.212  0.83223    
female:estier:PartyIDSPD    0.11122    0.08399   1.324  0.18556    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.4232 on 2498 degrees of freedom
Multiple R-squared:  0.0353,    Adjusted R-squared:  0.03183 
F-statistic: 10.16 on 9 and 2498 DF,  p-value: 1.704e-15

Option 2 (estimating interaction terms in separate models and keeping predictor variables)

lm(elected ~ female + estier + female:estier)
              Estimate Std. Error t value Pr(>|t|)    
(Intercept)    0.25070    0.01613  15.543   <2e-16 ***
female        -0.02006    0.02376  -0.844    0.399    
estier         0.01702    0.02205   0.772    0.440    
female:estier -0.04123    0.03586  -1.150    0.250    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.4298 on 2504 degrees of freedom
Multiple R-squared:  0.002427,  Adjusted R-squared:  0.001232 
F-statistic: 2.031 on 3 and 2504 DF,  p-value: 0.1075

lm(elected ~ female + estier + PartyID + female:estier:PartyID)
                           Estimate Std. Error t value Pr(>|t|)    
(Intercept)                 0.18082    0.02501   7.230 6.40e-13 ***
female                     -0.02850    0.02374  -1.201  0.23000    
estier                      0.02393    0.02189   1.093  0.27456    
PartyIDCDU                  0.07542    0.02704   2.789  0.00533 ** 
PartyIDGREEN                0.02731    0.02867   0.953  0.34088    
PartyIDSPD                  0.16666    0.02871   5.804 7.29e-09 ***
female:estier:PartyIDAfD   -0.12361    0.07676  -1.610  0.10742    
female:estier:PartyIDCDU    0.11820    0.05840   2.024  0.04307 *  
female:estier:PartyIDGREEN -0.14106    0.04724  -2.986  0.00286 ** 
female:estier:PartyIDSPD   -0.01240    0.05054  -0.245  0.80630    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.4232 on 2498 degrees of freedom
Multiple R-squared:  0.0353,    Adjusted R-squared:  0.03183 
F-statistic: 10.16 on 9 and 2498 DF,  p-value: 1.704e-15

Option 3 (including interaction terms without predictor variables in separate models)

lm(elected ~ female:estier)
               Estimate Std. Error t value Pr(>|t|)    
(Intercept)    0.251522   0.009303  27.037   <2e-16 ***
female:estier -0.045088   0.024123  -1.869   0.0617 .  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.4299 on 2506 degrees of freedom
Multiple R-squared:  0.001392,  Adjusted R-squared:  0.0009936 
F-statistic: 3.494 on 1 and 2506 DF,  p-value: 0.06173

lm(elected ~ female:estier:PartyID)
                            Estimate Std. Error t value Pr(>|t|)
(Intercept)                 0.251522   0.009231  27.247  < 2e-16
female:estier:PartyIDAfD   -0.198891   0.069807  -2.849  0.00442
female:estier:PartyIDCDU    0.118341   0.050769   2.331  0.01983
female:estier:PartyIDGREEN -0.189022   0.036724  -5.147 2.85e-07
female:estier:PartyIDSPD    0.078986   0.040337   1.958  0.05032
                              
(Intercept)                ***
female:estier:PartyIDAfD   ** 
female:estier:PartyIDCDU   *  
female:estier:PartyIDGREEN ***
female:estier:PartyIDSPD   .  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.4265 on 2503 degrees of freedom
Multiple R-squared:  0.01789,   Adjusted R-squared:  0.01632 
F-statistic:  11.4 on 4 and 2503 DF,  p-value: 3.622e-09

I have been running my analysis under the logic of Options 3, estimating interactions without the predictor variables. Given that the DF is approximately the same between the models, what is the substantive difference in the interaction coefficients between the models?

Is there something inherently wrong with excluding the predictor variables when estimating the interaction? Which of the three options would be best for my aim?

For reference, female and estier are binary variables, and PartyID is categorical, with four categories.

Answer 1

When running a multiple regression model with interacting terms, you will want to include all of the main effects comprising those interactions in your model. Furthermore, if you have 3-way interactions, you should include all the main effects along with all of the sub-2-way interactions.

I recommend running the model with the three-way interaction and all sub interactions: lm(elected ~ female * estier * PartyID). Because you have a categorical variable, I would look at the anova(·) of this lm(·) output to determine if the 3-way interaction is indeed statistically significant. If not, then you can remove it and focus on just the 2-way interactions.

Happy to elaborate more if needed.

---

Update #1
Like polynomial predictors, interacting predictors really only depend on their highest powered term or terms. In fact, with a point-centering transformation, you can change the parameters of your lower termed powers to just about anything you might want. Thus, the focus is the highest power.

However, the behavior of that higher power may be obscured if the lower powers aren't actually included. Said another way, what you might be seeing with an interaction-only model is just the linear effect of one or both of the predictors "sneaking" through.

Here is a small simulation that demonstrates this:

set.seed(1234)
n <- 25
eps <- rnorm(n,0,15)
x <- rnorm(n,50,10)
y <- 100 - 1.2 * x + eps
z <- 10 + 2 * x - 3 * y + 0.00 * x*y + rnorm(n,0,10)
summary(lm(z ~ x * y))  # proper model, intxn n.s.
summary(lm(z ~ x : y))  # incorrect model, int'n stat.sig.

So, you want to include all the main-effects whenever you include an interaction.

Hope this helps.