# regression models and dummy variables

I have a output variable and 1 categorical predictor and 3 continuous predictors.

dat=data.frame(IV = rnorm(9), group=c("A","A","A","B","B","B","C","C","C"),pred1=rnorm(9),pred2=rnorm(9),pred3=rnorm(9))


What is the recommended way to add the dummy variable and how does the interpretation of the output change with these 2 setups?

If I do this and include a single column for the dummy variable

  lm(IV~factor(group)+pred1+pred2+pred3, data = dat)

lm(formula = IV ~ factor(group) + pred1 + pred2 + pred3, data = dat)

Coefficients:
(Intercept)  factor(group)B  factor(group)C           pred1           pred2           pred3
-0.3143          0.9496         -0.3502          0.1839         -0.5349         -0.4710


In the case above the results the groupB and groupC coefficients are RELATIVE to groupA.

OR if I don't want the results to be relative I can include groupA like this:

dat2 = cbind(dat[,c("IV","pred1","pred2","pred3")], model.matrix(~ factor(dat$group) - 1) ) lm(IV~., data = dat2) Call: lm(formula = IV ~ ., data = dat2) Coefficients: (Intercept) pred1 pred2 pred3 factor(dat$group)A  factor(dat$group)B factor(dat$group)C
-0.6645                0.1839               -0.5349               -0.4710                0.3502                1.2998                    NA


I have a few questions:

1. How does the interpretation of the intercept change between the 2 models?

2. Why in the last model is the GROUP C coefficient "NA"?

• In what sense your last model is a "fixed effect" model that the first two are not? Or, do you mean something else? I don't think you are using three different setups. Your first two models are identical; group is already a factor, using factor(group) is redundant. – T.E.G. Jan 11 '17 at 0:37
• See update I am looking for interpetation of intercept and the NA issue – user3022875 Jan 11 '17 at 0:41
• I think in your last model, you are falling into what is called dummy variable trap. See Andy's answer here (first paragraph): stats.stackexchange.com/questions/144372/dummy-variable-trap . You need to drop one dummy as reference. R did that automatically in the first model. You forced all dummies into the model in the second one. I guess that is why you have NA and a different intercept. – T.E.G. Jan 11 '17 at 0:53

## 1 Answer

In your first model, the intercept is the expected value for group A when pred1, pred2, and pred3 are all 0. The coefficients for group B and group C are the expected difference from Group A for these other groups (at any value of the pred vars as you have no interactions).

(Your example is not fully reproducible as you did not set seed. So I repeat with a fixed seed.)

> set.seed(20190727-2150)
>
> dat=data.frame(IV = rnorm(9), group=c("A","A","A","B","B","B","C","C","C"),pred1=rnorm(9),pred2=rnorm(9),pred3=rnorm(9))
> lm(IV ~ group + pred1 + pred2 + pred3, data=dat)

Call:
lm(formula = IV ~ group + pred1 + pred2 + pred3, data = dat)

Coefficients:
(Intercept)       groupB       groupC        pred1        pred2
0.58421      0.65581     -1.22962     -0.14758      0.50855
pred3
0.02724


In your second model, you forced all groups into the model, but also included an intercept. This leads to perfect collinearity so there is no single solution to the regression equations. lm dropped one of the offending variables (group C) to fit the model. This is why the coefficient for group C is NA.

> dat2 = cbind(dat[,c("IV","pred1","pred2","pred3")], model.matrix(~ factor(dat$group) - 1) ) > lm(IV~., data = dat2) Call: lm(formula = IV ~ ., data = dat2) Coefficients: (Intercept) pred1 pred2 -0.64541 -0.14758 0.50855 pred3 factor(dat$group)A  factor(dat$group)B 0.02724 1.22962 1.88543 factor(dat$group)C
NA


So now the intercept is the expected value for group C when pred1, pred2, and pred3 are all 0. The coefficients for group A and group B are the expected difference from Group C for these other groups (at any value of the pred vars as you have no interactions).

Note that the intercept in the second model is the same as the intercept in the first model plus the coefficient for group C.

> .58421 + -1.22962
[1] -0.64541


I think for your second model, you intended to exclude the intercept.

> lm(IV ~ 0 + group + pred1 + pred2 + pred3, data=dat)

Call:
lm(formula = IV ~ 0 + group + pred1 + pred2 + pred3, data = dat)

Coefficients:
groupA    groupB    groupC     pred1     pred2     pred3
0.58421   1.24002  -0.64541  -0.14758   0.50855   0.02724


Now there is no intercept (or 3 intercepts, for groups A, B, C, respectively).

The coefficients for each group are the expected values for those groups when pred1, pred2, and pred3 are all 0.

You can see the coefficient for group A is the same as the intercept in your first model. You can see the coefficient for group C is the same as the intercept in your second model.

The coefficient for group B is the same as:

Model 1: Intercept + group B coefficient:

> .58421 + .65581
[1] 1.24002


Model 2: Intercept + group B coefficient:

> -.64541 + 1.88543
[1] 1.24002


I think I have answered your specific questions about interpretation of the intercept and reason for NA in the second model.

Your initial question was "What is the recommended way to add the dummy variable "

As you can see from above, all the models are equivalent. Add the dummies in the way that facilitates the interpretation you want. If you care about differences from a reference group, use the first model. If you are mainly interested in expected values in each group, you can drop the intercept and include all dummies.

But don't try to include all dummies and an intercept! (unless you really really know what you are doing and why you want to do it).