What does the regression model look like if the predictors are categorical?

Question

I was reading "The lm() function with categorical predictors", and am confused.

1. What does the regression model with a categorical predictor look like, with the following R code:

   n = 30
   sigma = 2.0
   
   AOV.df <- data.frame(category = c(rep("category1", n)
                                   , rep("category2", n)
                                   , rep("category3", n)), 
                               j = c(1:n
                                   , 1:n
                                   , 1:n),
                               y = c(8.0  + sigma*rnorm(n)
                                   , 9.5  + sigma*rnorm(n)
                                   , 11.0 + sigma*rnorm(n))
                     )
   
   AOV.lm <- lm(y ~ category, data = AOV.df)
   summary(AOV.lm)
   

   The output is:

   Coefficients:
                     Estimate Std. Error t value Pr(>|t|)    
   (Intercept)         8.4514     0.3405  24.823  < 2e-16 ***
   categorycategory2   0.8343     0.4815   1.733   0.0867 .  
   categorycategory3   3.0017     0.4815   6.234 1.58e-08 ***
   ---
   Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 
   
   Residual standard error: 1.865 on 87 degrees of freedom
   Multiple R-squared: 0.3225, Adjusted R-squared: 0.307 
   F-statistic: 20.71 on 2 and 87 DF,  p-value: 4.403e-08 
   

   Is the model:

   y = 8.4514 + 0.8343 for category 1,

   y = 8.4514 + 0.8343 for category 2 or

   y = 8.4514 + 3.0017 for cateogry 3?

2. What is the model if the R code looks like:

   > X <- read.table("http://www.stat.umn.edu/geyer/5102/data/ex5-4.txt", header=T) 
   > lm(y ~ color + x * color, data=X)
   
   Call:
   lm(formula = y ~ color + x * color, data = X)
   
   Coefficients:
    (Intercept)    colorgreen      colorred             x  colorgreen:x  
       13.96118       0.25243       6.10543       0.97241       0.07347  
     colorred:x  
        0.01962
   

Thanks!

Answer 1

1. Your model is this:

   y = 8.4514 + (0.8343 * category2) + (3.0017 * category3)

   where category2 is an indicator (0/1 "dummy" variable) that is 1 when category==2 and 0 otherwise and where category3 is an indicator (0/1 "dummy" variable) that is 1 when category==3 and 0 otherwise. The baseline level of the category variable is dropped due to multicollinearity.

2. You'll see something similar if you include an interaction term, with one level of the categorical variable excluded and interaction terms between x and the remaining category levels. The coefficient on x will represent the marginal effect of x when category==1 and the interaction terms will be the marginal effects of x when category==2 and category==3, respectively. The indicators categorycategory2 and categorycategory3 will take on a slightly different meaning, as well, being the marginal effects of the category2 and category3 indicators when x==0.

Also, as a side note: Since you have a model with all indicator variables, you may also find the no-intercept model intuitive (since it looks like you're working from an ANOVA mindset), in which case the coefficients are group means:

> summary(lm(y ~ 0 + category, data = AOV.df))

                  Estimate Std. Error t value Pr(>|t|)    
categorycategory1   8.1714     0.3247   25.16   <2e-16 ***
categorycategory2   9.4185     0.3247   29.00   <2e-16 ***
categorycategory3  10.8858     0.3247   33.52   <2e-16 ***

Answer 2

lm(y ~ color + x * color, data = X)

is redundant because "x * color" is interpreted as "x + color + x:color" where "x:color" is the interaction between x and color.

lm(y ~ x * color, data = X)

is enough. The model is: $$y = \beta_0+\beta_1 x+\beta_2 color + \beta_3 x\,color + \varepsilon$$ Because color is a three-level factor, you can read it as: $$y=\beta_0+\beta_1 x+\beta_{2g}(color = green)+\beta_{2r}(color=red)+\beta_{3g}x(color=green)+\beta_{3r}x(color=red)+\varepsilon$$ i.e.:

• when color is "blue": $y=13.96+0.97x$;
• when color is "green": $y=13.96+0.97x+0.25+0.07x=(13.96+0.25)+(0.97+0.07)x$;
• when color is "red": $y=13.96+0.97x+6.11+0.02x=(13.96+6.11)+(0.97+0.02)x$.