I learned that in order to use categorical variables in Linear Regression models, I have to convert them to several binary dummy variables. Binary dummy variables can either be 0 or 1, so they aren't continuuos, too. So why does it work better with binary dummies?

  • $\begingroup$ for dummy coding you convert k categories into a k-1 dummy variables, so your binary variable has 2 categories you will get 1 binary dummy that is either 0 or 1. I don't know why would you think one works better than another one, they all work the same $\endgroup$ – rep_ho Jun 19 at 9:52
  • $\begingroup$ Dummy variables or other contrast variables are (used as) quantitative. That is important. It doesn't matter if they are integer-valued or fractional-valued. Simple contrast variables are fractions, for example. All contrast variable types are discrete, that is, they can bear a limited amount of possible values, but the amount can be more than two or three. $\endgroup$ – ttnphns Jun 19 at 10:21
  • $\begingroup$ About contrasts and contrast variables of different types stats.stackexchange.com/a/221868/3277 $\endgroup$ – ttnphns Jun 19 at 10:26

It is all about interpretation.

Suppose you have a variable $X$ that can take three different values: A, B and C. And suppose we are interested only in the realtionship between $X$ and some response variable $Y$. If we do not encode $X$ with the dummy variables, but use $0$, $1$ and $2$ instead, then the model is:

$$ Y = \alpha + \beta X + \epsilon.$$

When we estimate $\beta$ with $\hat{\beta}$, then our interpretation of the model would be something like: "a change in X from group A to group B is associated with $\hat{\beta}$ changes in units of $Y$, whereas a change in X from group A to group C is associated with twice as much change!"

But what if you code the groups differently? For example, you can change $A$ and $B$, and you will get a completely different interpretation. The estimator may even be very close, but the conslusion will be different.

With dummy variables you wouldn't run into these kind of problems.

There are models where it makes sense: for example, if you have $X$ representing age groups. If the categorical variable is ordered and one expects that the difference between A and B is the same as between B and C, then yes, it may make sense to just forget about dummies and use the variables straightforward. But most often, the categorical variables stand for something like race or gender, where there is no ordering. Hence, dummy variables.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.