3
$\begingroup$

I'd like to perform logistic regression with some categorical explanatory variables with more categories than just binary 0/1. Is this possible and why?

I am inclined to think that this would give just the wrong result because of the geometric intuition, however most opinions online say that explanatory variables can be discrete/categorical (although I don't see any mentions of more categories). For 0/1 this is fine because the distances between 0 and 1 is easy.

If I have even 3 categories 0/1/2, it's unclear if 1-0 is the same as 2-1.

One thought I had is to do a sort of one-vs-all thing for the explanatory variables. So if I had one parameter that can on 3 categories A/B/C I remake this into 3 separate parameters: 

A: 0/1
B: 0/1
C: 0/1

Where 0 is not belonging to the letter class and 1 is belonging to the letter class.

Background: My dependent variable should be binary. I really wanted to use logistic regression because of the probability interpretation. If you can recommend another method that can output a probability estimate (instead of only classification) that takes in categorical explanatory variables, that would also be helpful.

Improve this question
$\endgroup$
5
  • 2
    $\begingroup$ It's fine: logistic regression requires only that the logit of the probability of "success" be a sum of parameters multiplied by explanatory variables, where the explanatory variables can be anything you like - $x_1$, $x_1^2$, $x_1x_2$, splines, &c. The only thing wrong with your coding scheme is that everything has to belong to some category, so you don't need three parameters for the categorical variable in addition to an intercept - take say A as the reference level, & then parameters for B & C represent changes in the logit going from A to B, or from A to C. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Commented Jul 31, 2014 at 17:22
  • $\begingroup$ Yes, thanks for catching that typo. Fixed in the question. $\endgroup$
    – jyfan
    Commented Jul 31, 2014 at 17:24
  • $\begingroup$ .I'm not sure I understand what you mean by the "reference level". An example would be I have a explanatory variable "color" which can be "blue", "red", "green". My coding scheme basically says that I would change this into 3 binary parameters: red/notred, blue/notblue and green/notgreen. Assuming that only these 3 colors exist in the world. $\endgroup$
    – jyfan
    Commented Jul 31, 2014 at 17:28
  • 2
    $\begingroup$ I'm off out, but Google "coding categorical variables" or look here, or wait for a proper answer. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Commented Jul 31, 2014 at 17:31
  • 2
    $\begingroup$ "My coding scheme basically says that I would change this into 3 binary parameters: red/notred, blue/notblue and green/notgreen. Assuming that only these 3 colors exist in the world." Correct. But when fitting the model, you'll only need two of the three colors. The one that didn't get in is the reference category, it will be captured by the intercept. $\endgroup$
    – Penguin_Knight
    Commented Jul 31, 2014 at 17:32

1 Answer 1

Reset to default
3
$\begingroup$

The comments from @Scortchi and @Penguin_Knight essentially say it all. Any generalized linear model can handle categorical predictors, and several options exist to accommodate different styles of comparison. Dummy coding is probably simplest, and would only require you to drop one of your binary parameters corresponding to your choice of a reference group. Scortchi's link to the UCLA Statistical Consulting Group's R Library: Contrast Coding Systems for categorical variables page covers this and many other coding options nicely with code included.

Sorry I don't have more to add; it seems you already understand the method, and I don't know of any other methods to recommend for your specific purpose, nor do I see the problem with the geometric intuition...Feel free to expand on that though.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.