If I have a variable with 4 levels, in theory I need to use 3 dummy variables. In practice, how is this actually carried out? Do I use 0-3, do I use 1-3 and leave the 4's blank? Any suggestions?

NOTE: I'm going to be working in R.

UPDATE: What would happen if I just use one column that uses 1-4 corresponding to A-D? Will that work or introduce problems?

  • 3
    $\begingroup$ I think this page from UCLA ATS explains it quite well. $\endgroup$ – caracal Dec 22 '11 at 17:00
  • 4
    $\begingroup$ Just to be clear, note that coding this variable as integers 0-3 or 1-4 or 1-3 is not a dummy coding and will not have the same effect as three dummy variables. However, such an incorrect coding will work in regression formulas and software and there will be plausible output: it just won't correspond to the intended model. (NB: this answers the recent update to the question.) $\endgroup$ – whuber Dec 22 '11 at 17:13

In practice, one usually lets one's software of choice handle creating and manipulating the dummy variables. There are several ways it might be handled; here are several common possibilities for a data set with four observations, one at each level of A, B, C, and D. These are different parameterizations; they result in exactly the same model fit, but with different interpretations to the parameters. One can easily convert from one to another using basic algebra; note they are all linear combinations of each other; in fact, any linear combination can be used.

Use differences from the first level (default in R):

A 0 0 0
B 1 0 0
C 0 1 0
D 0 0 1

Use differences from the last level (default in SAS):

A 1 0 0
B 0 1 0
C 0 0 1
D 0 0 0

Use "sum" contrasts:

A    1    0    0
B    0    1    0
C    0    0    1
D   -1   -1   -1

Use "helmert" contrasts:

A   -1   -1   -1
B    1   -1   -1
C    0    2   -1
D    0    0    3
| cite | improve this answer | |
  • 1
    $\begingroup$ I'd say, for the sake of purity, that dummy variables, in the strict sence, is only 1st and 2nd your examples. Dummy variables are also known as indicator contrast variables. Helmert, deviation and other alternative types of contrast variables shouldn't be called dummy, for me. $\endgroup$ – ttnphns Dec 23 '11 at 10:15
  • 1
    $\begingroup$ @ttnphns I agree that Helmert contrasts don't constitute dummy coding from a technical standpoint, but I think it's perfectly reasonable for them to be included here nonetheless. I can't tell if you are pointing this out for the sake of clarity or suggesting that the answer be changed. @ Aaron +1, this answer would be even better if you were to explain briefly how the interpretation of these these different coding schemes would differ. $\endgroup$ – gung - Reinstate Monica Dec 24 '11 at 17:20

Let us assume your variable levels are A, B, C, and D. If you have a constant term in the regression, you need to use three dummy variables, otherwise, you need to have all four.

There are many mathematically equivalent ways you can implement the dummy variables. If you have a constant term in the regression, one way is to pick one of the levels as the "baseline" level and compare the other three to it. Let us say, for concreteness, that the baseline level is A. Then your first dummy variable takes on the value 1 whenever the level is B and 0 otherwise; the second takes on the value 1 whenever the level is C and 0 otherwise, and the third takes on the value 1 whenever the level is D and 0 otherwise. Because your constant term is equal to 1 all the time, the first dummy variable's estimated coefficient will be the estimate of the difference between level B and A, and similarly for the other dummy variables.

If you don't have a constant term, you can just use four dummy variables, constructed as in the previous example, just adding one for the A level.

| cite | improve this answer | |
  • $\begingroup$ Nice note on how having a constant term in the regression matters. $\endgroup$ – Aaron left Stack Overflow Dec 22 '11 at 23:17

In R, define the variable as a factor and it will implement it for you:

x <- as.factor(sample(LETTERS[1:4], 20, replace = TRUE))
y <- rnorm(20)
lm (y ~ x)

which returns

lm(formula = y ~ x)

(Intercept)           xB           xC           xD  
     1.0236      -0.6462      -0.9466      -0.4234  

The documentation for 'lm', 'factor', and 'formula' in R fills in some of the details.

| cite | improve this answer | |
  • 1
    $\begingroup$ +1 This is a nice complement to the answers already listed. We can also note that if you already have a variable with group names (such as A-D), this can be done in the analysis function call without an extra step: lm(y ~ as.factor(x)) $\endgroup$ – gung - Reinstate Monica Dec 24 '11 at 17:13
  • $\begingroup$ The main reason I'm looking at using dummy variables is that I'm working with a large data set with many factor levels (>32) and some packages in R (namely randomforest) cannot handle factors with many levels, so I was trying to see if dummy's were a work around. $\endgroup$ – screechOwl Dec 26 '11 at 15:47
  • 1
    $\begingroup$ You can construct the regression design matrix using 'model.matrix': model.matrix(y ~ x) (x is still a factor) will give you a matrix with the dummy variables. I'm not familiar with the randomforest package, but I suspect that you can give give any functions an explicit design matrix that you get from model.matrix, and model.matrix seems to work with many (ie hundreds) of levels. $\endgroup$ – Gray Dec 26 '11 at 20:26
  • $\begingroup$ ps: you may want to edit the question to make it clear that you care about solutions that work with lots of levels. The question asks about 4. $\endgroup$ – Gray Dec 26 '11 at 20:33

whuber told you in the comments that coding a 0-3 or 1-4 coding instead of creating dummy variables isn't what you want. This is try - I am to hopefully explain what you would be doing with that model and why it is wrong.

If you do code a variable X such that if A then X=1, if B then X=2, if C then X=3, if D then X=4 then when you do the regression you'll only get one parameter. Let's say it ended up be that the estimated parameter associated with X was 2. This would tell you that the expected difference between the mean of B and the mean of A is 2. It also tells you that the expected difference between the mean of C and the mean of B is 2. Some for D and C. You would be forcing the differences in the means for these groups to follow this very strict pattern. That one parameter tells you exactly how all of your group means relate to each other.

So if you did this kind of coding you would need to assume that not only did you get the ordering correct (because in this case if you expect an increase from A to B then you need to expect an increase from B to C and from C to D) but you also need to assume that that difference is the same!

If instead you do the dummy coding that has been suggested you're allowing each group to have its own mean - no restrictions. This model is much more sensible and answers the questions you want.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.