0
$\begingroup$

I am testing out the lm function for multiple linear regression. One of my predictors, X, is categorical and has 2 levels (there are 2 labels that the observation could take on for this variable). Then I have some numeric predictors, and another categorical predictor, Y, which has levels (3 different labels that each observation could take on for this variable). When I use lm, variable X just shows up in the summary with one coefficient as I would expect, but then there are 2 coefficients in the model for variable Y, which are written as YName1 and YName2 (and not for name3). Why is this? Shouldn't there only be one coefficient for each predictor, and why is variable Y any different from variable X? My model is just specified as

model <- lm(dependent ~ X + ... + Y, data=data)

Any help would be appreciated.

Improve this question
$\endgroup$
1

2 Answers 2

Reset to default
0
$\begingroup$

Actually when you have a categorical predictor such as yours, coefficients are computed regarding a reference value. Thus, if Y can be "a" "b" or "c" , you have coefficients for "b" and for "c", no coefficients standing for a "baseline" value which is Y = "a".

You can do it differently by constructing 3 "dummy variables" mutually exclusives

  • Ya = 0 if non a and 1 if a
  • Yb = 0 if non b and 1 if b
  • etc.

Hope I have helped :)

Improve this answer
$\endgroup$
4
  • $\begingroup$ I see, so since variable X only has two levels it only needs one dummy variable right? $\endgroup$
    – fmtcs
    Dec 5, 2021 at 14:58
  • $\begingroup$ If I want to test the model with/without the variable Y, do I remove the entire variable Y or do I remove one level of Y at a time? $\endgroup$
    – fmtcs
    Dec 5, 2021 at 15:02
  • $\begingroup$ Exactly : X is a binary variable so it counts for one dummy variable. $\endgroup$
    – Symphonic
    Dec 5, 2021 at 15:18
  • $\begingroup$ I think you should remove the entire variable but it depends on what you want to test. $\endgroup$
    – Symphonic
    Dec 5, 2021 at 15:19
2
$\begingroup$

Some additional information to help you to move forward with categorical variable...

When dealing with categorical variable (which can be enforced using factor() to make sure that R does not treat it as another type), the reference is actually included in the intercept:

require(datasets); data(InsectSprays)
model1 <- lm(count ~ spray, data = InsectSprays) 
summary(model1)$coefficients

##                Estimate Std. Error    t value     Pr(>|t|)
## (Intercept)  14.5000000   1.132156 12.8074279 1.470512e-19
## sprayB        0.8333333   1.601110  0.5204724 6.044761e-01
## sprayC      -12.4166667   1.601110 -7.7550382 7.266893e-11
## sprayD       -9.5833333   1.601110 -5.9854322 9.816910e-08
## sprayE      -11.0000000   1.601110 -6.8702352 2.753922e-09
## sprayF        2.1666667   1.601110  1.3532281 1.805998e-01

So, here, spray A is included in the intercept and its value is 14.5. Since it is the reference, all the other coefficients are given based on this value (they are compared with the reference sprayA). For instance, the coefficient for sprayB is 14.5 + 0.833 = 15.333. The coefficient for sprayC is 14.5 - 12.416 = 2.084.

If you want to get the coefficient value of each spray category, you can also do the following (subtracting by 1) which remove the intercept:

model2 <- lm(count ~ spray - 1, data = InsectSprays) 
summary(model2)$coefficients

##         Estimate Std. Error   t value     Pr(>|t|)
## sprayA 14.500000   1.132156 12.807428 1.470512e-19
## sprayB 15.333333   1.132156 13.543487 1.001994e-20
## sprayC  2.083333   1.132156  1.840148 7.024334e-02
## sprayD  4.916667   1.132156  4.342749 4.953047e-05
## sprayE  3.500000   1.132156  3.091448 2.916794e-03
## sprayF 16.666667   1.132156 14.721181 1.573471e-22

Which is equivalent to what we calculated manually earlier.

Finally, if you want to change your reference, you can use the relevel() function:

sprayDRef <- relevel(InsectSprays$spray, "D")
model3 <- lm(count ~ sprayDRef, data = InsectSprays) 
summary(model3)$coefficients

##              Estimate Std. Error   t value     Pr(>|t|)
## (Intercept)  4.916667   1.132156  4.342749 4.953047e-05
## sprayDRefA   9.583333   1.601110  5.985432 9.816910e-08
## sprayDRefB  10.416667   1.601110  6.505905 1.212803e-08
## sprayDRefC  -2.833333   1.601110 -1.769606 8.141205e-02
## sprayDRefE  -1.416667   1.601110 -0.884803 3.794750e-01
## sprayDRefF  11.750000   1.601110  7.338660 4.035610e-10

Here, all the different spray category coefficients are compared with spray D.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for the answer, how do I know whether R is treating the variable as a type or not? When I use factor() on the variable in the lm statement (which I read you can do), it says 1: the response appeared on the right-hand side and was dropped 2: Problem with term in model.matrix: no columns are assigned $\endgroup$
    – fmtcs
    Dec 5, 2021 at 18:25
  • $\begingroup$ If your variable is categorical but coded as numeric (for instance, if in the example above, spray (A, B, C... F) is coded with (1,2,3...,6)), R will take it as numeric. You can check the type of the variable with the command class(). Not sure to understand the second part of your problem... Is this what you get when you run model <- lm(dependent ~ factor(X) + ... + Y, data=data) (if I take your original example)? $\endgroup$
    – Pitouille
    Dec 5, 2021 at 19:51

Not the answer you're looking for? Browse other questions tagged or ask your own question.