regression models and dummy variables

Question

I have a output variable and 1 categorical predictor and 3 continuous predictors.

dat=data.frame(IV = rnorm(9), group=c("A","A","A","B","B","B","C","C","C"),pred1=rnorm(9),pred2=rnorm(9),pred3=rnorm(9))

What is the recommended way to add the dummy variable and how does the interpretation of the output change with these 2 setups?

If I do this and include a single column for the dummy variable

  lm(IV~factor(group)+pred1+pred2+pred3, data = dat)

lm(formula = IV ~ factor(group) + pred1 + pred2 + pred3, data = dat)

Coefficients:
   (Intercept)  factor(group)B  factor(group)C           pred1           pred2           pred3  
       -0.3143          0.9496         -0.3502          0.1839         -0.5349         -0.4710  

In the case above the results the groupB and groupC coefficients are RELATIVE to groupA.

OR if I don't want the results to be relative I can include groupA like this:

dat2 = cbind(dat[,c("IV","pred1","pred2","pred3")], model.matrix(~ factor(dat$group) - 1) )
lm(IV~., data = dat2)

Call:
lm(formula = IV ~ ., data = dat2)

Coefficients:
         (Intercept)                 pred1                 pred2                 pred3  `factor(dat$group)A`  `factor(dat$group)B`  `factor(dat$group)C`  
             -0.6645                0.1839               -0.5349               -0.4710                0.3502                1.2998                    NA  

I have a few questions:

1. How does the interpretation of the intercept change between the 2 models?

2. Why in the last model is the GROUP C coefficient "NA"?

Answer 1

In your first model, the intercept is the expected value for group A when pred1, pred2, and pred3 are all 0. The coefficients for group B and group C are the expected difference from Group A for these other groups (at any value of the pred vars as you have no interactions).

(Your example is not fully reproducible as you did not set seed. So I repeat with a fixed seed.)

> set.seed(20190727-2150)
> 
> dat=data.frame(IV = rnorm(9), group=c("A","A","A","B","B","B","C","C","C"),pred1=rnorm(9),pred2=rnorm(9),pred3=rnorm(9))
> lm(IV ~ group + pred1 + pred2 + pred3, data=dat)

Call:
lm(formula = IV ~ group + pred1 + pred2 + pred3, data = dat)

Coefficients:
(Intercept)       groupB       groupC        pred1        pred2  
    0.58421      0.65581     -1.22962     -0.14758      0.50855  
      pred3  
    0.02724  

In your second model, you forced all groups into the model, but also included an intercept. This leads to perfect collinearity so there is no single solution to the regression equations. lm dropped one of the offending variables (group C) to fit the model. This is why the coefficient for group C is NA.

> dat2 = cbind(dat[,c("IV","pred1","pred2","pred3")], model.matrix(~ factor(dat$group) - 1) )
> lm(IV~., data = dat2)

Call:
lm(formula = IV ~ ., data = dat2)

Coefficients:
         (Intercept)                 pred1                 pred2  
            -0.64541              -0.14758               0.50855  
               pred3  `factor(dat$group)A`  `factor(dat$group)B`  
             0.02724               1.22962               1.88543  
`factor(dat$group)C`  
                  NA  

So now the intercept is the expected value for group C when pred1, pred2, and pred3 are all 0. The coefficients for group A and group B are the expected difference from Group C for these other groups (at any value of the pred vars as you have no interactions).

Note that the intercept in the second model is the same as the intercept in the first model plus the coefficient for group C.

> .58421 + -1.22962
[1] -0.64541

I think for your second model, you intended to exclude the intercept.

> lm(IV ~ 0 + group + pred1 + pred2 + pred3, data=dat)

Call:
lm(formula = IV ~ 0 + group + pred1 + pred2 + pred3, data = dat)

Coefficients:
  groupA    groupB    groupC     pred1     pred2     pred3  
 0.58421   1.24002  -0.64541  -0.14758   0.50855   0.02724  

Now there is no intercept (or 3 intercepts, for groups A, B, C, respectively).

The coefficients for each group are the expected values for those groups when pred1, pred2, and pred3 are all 0.

You can see the coefficient for group A is the same as the intercept in your first model. You can see the coefficient for group C is the same as the intercept in your second model.

The coefficient for group B is the same as:

Model 1: Intercept + group B coefficient:

> .58421 + .65581
[1] 1.24002

Model 2: Intercept + group B coefficient:

> -.64541 + 1.88543
[1] 1.24002

I think I have answered your specific questions about interpretation of the intercept and reason for NA in the second model.

Your initial question was "What is the recommended way to add the dummy variable "

As you can see from above, all the models are equivalent. Add the dummies in the way that facilitates the interpretation you want. If you care about differences from a reference group, use the first model. If you are mainly interested in expected values in each group, you can drop the intercept and include all dummies.

But don't try to include all dummies and an intercept! (unless you really really know what you are doing and why you want to do it).