Which groups are reference groups in a regression model with interaction?

Question

What are the reference groups in a regression model where there are interaction categories? Using the iris dataset in R, I've created a category with three levels from the Sepal.Width variable, and built a linear model with interaction:

data("iris")

iris$SW_cat = as.factor(ifelse(iris$Sepal.Width<3,0,
                     ifelse(iris$Sepal.Width<3.5,1,2)))

model1 = lm(data=iris, Sepal.Length ~ Species +
              SW_cat + Species*SW_cat)

summary(model1) returns the parameters from the model:

summary(model1)
#> 
#> Call:
#> lm(formula = Sepal.Length ~ Species + SW_cat + Species * SW_cat, 
#>     data = iris)
#> 
#> Residuals:
#>     Min      1Q  Median      3Q     Max 
#> -1.4381 -0.2539 -0.0381  0.2414  1.3619 
#> 
#> Coefficients: (1 not defined because of singularities)
#>                           Estimate Std. Error t value Pr(>|t|)    
#> (Intercept)                4.45000    0.32564  13.666  < 2e-16 ***
#> Speciesversicolor          1.35294    0.33508   4.038 8.79e-05 ***
#> Speciesvirginica           1.88810    0.34079   5.540 1.42e-07 ***
#> SW_cat1                    0.40385    0.33793   1.195   0.2341    
#> SW_cat2                    0.78636    0.34012   2.312   0.0222 *  
#> Speciesversicolor:SW_cat1  0.01196    0.36564   0.033   0.9739    
#> Speciesvirginica:SW_cat1  -0.06886    0.36394  -0.189   0.8502    
#> Speciesversicolor:SW_cat2       NA         NA      NA       NA    
#> Speciesvirginica:SW_cat2   0.47554    0.44325   1.073   0.2852    
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#> 
#> Residual standard error: 0.4605 on 142 degrees of freedom
#> Multiple R-squared:  0.7052, Adjusted R-squared:  0.6907 
#> F-statistic: 48.54 on 7 and 142 DF,  p-value: < 2.2e-16

Notably, the output is missing the following interactions:

• Speciessetosa:SW_cat0
• Speciessetosa:SW_cat1
• Speciessetosa:SW_cat2

Does this mean there are three reference groups in such an interaction model? I'm not sure how this can be the case, if a reference group is simply what the other terms are being compared to. Further, what is the interpretation of the intercept, give the model cannot take all three of these reference values at once?

Answer 1

Every categorical variable has a reference group. Speciessetosa is the reference group for Species and SW_cat0 is the reference group for SW_cat.

The Sepal.Length for Speciessetosa:SW_cat0 is the same as the Intercept (because both are reference groups, they are 'collapsed' into the intercept value).

The Sepal.Length for Speciessetosa:SW_cat1 is the same as Intercept + the main effect SW_cat1.

The Sepal.Lengthfor Speciessetosa:SW_cat2 is the same as Intercept + the main effect SW_cat2.

Plotting the model results makes this much easier to follow.

Answer 2

• note: I'm a VERY beginner, but I'll try to give you a hint. (if I'm wrong, please any moderator feel free to exclude this comment).

Whenever I have a complex model, I try to do the artimetic by hand, I find it really helpful. So, I'd begin with thinking that whenever we have a categorical variable with n levels, we would have n-1 coefficients for that variable.

• What's the reference level?

### The reference will be what appears first using levels():

levels(iris$SW_cat) ### 3 levels
levels(iris$Species) ### 3 levels

### Output:

> levels(iris$SW_cat) ### 3 levels
[1] "0" "1" "2"
> levels(iris$Species) ### 3 levels
[1] "setosa"     "versicolor" "virginica" 

### note that you can change the reference levels with fct_relevel()

• How are they coded?

Well, if you haven't made the constrasts yourself, then you should have the dummy treatment constrast as this for both variables: this link sounds helpful on visualizing it

contr.treatment(3)

### output: 

 2 3
1 0 0
2 1 0
3 0 1

With this in mind, you can think of what combinations refers to each levels of your variable (i.e, the X values in the equation), and the video linked below may help with it as well as the linked post above :)

• What I'd do next is to try to come up with the equation myself and then I'd plug in zeros and ones (dummy coding) to get a better perception of what's actually happening in the data.

this video seems really helpful to understand the coefficient coding

y = b0 + b1X1 + b1X1,2 + b2X2 + b2X2,2 + the interaction coefficients

Here I stop because it's beyond my knowledge and I wanna help, not confuse you. So I'd take a look at the first answer to this postin order to try to better understand how to write down the interaction parameters. My guts tell me that you'd have two: b3(X1 * X2) and b3 (X1,2 * X2,2). But I'm not sure about that, I'm sorry, but I hope this is at least a hint for you :)

Last but not least, this video deals with 2-level variables, but it has changed my life on how to interpret categorical*categorical interactions, so it may be helpful too