Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have to do binary logistic regression with a lot of independent variables. Most of them are binary, but a few of the categorical variables have more than two levels.

What is the best way to deal with such variables?

For example, for a variable with three possible values, I suppose that two dummy variables have to be created. Then, in a step-wise regression procedure, it is better to test both of the dummy variables at the same time, or to test them separately?

I will use SPSS, but I do not remember it very well, so: how does SPSS deal with this situation?

Moreover, for an ordinal categorical variable, it is a good thing to use dummy variables which recreate the ordinal scale? (For example, using three dummy variables for a 4-state ordinal variable, put 0-0-0 for level $1$, 1-0-0 for level $2$, 1-1-0 for level $3$ and 1-1-1 for level $4$, instead of 0-0-0, 1-0-0, 0-1-0 and 0-0-1 for the 4 levels.)

share|improve this question
2  
This is just a partial answer: even when you create the dummies explicitly (rather than using the software's implicit capabilities), keep them together in all analyses. In particular, they should all enter together and all leave together in a stepwise regression, with the p-value computed appropriately for the total number of variables involved. (This is Hosmer & Lemeshow's recommendation, anyway, and it makes a lot of sense.) –  whuber Oct 7 '10 at 20:53
    
I wrote a post a while back on multinomial logistic regression resources in SPSS. –  Jeromy Anglim Oct 8 '10 at 5:44
    
You're talking about your independent variables. It's only the dependent variables that need to be binary for logistic regression. –  John Aug 30 '12 at 13:43
1  
One thing to note here is that you shouldn't use stepwise selection procedures at all; they are not valid. If that doesn't make sense / you want to understand why, it may help you to read my answer here: algorithms-for-automatic-model-selection. –  gung Jul 18 '13 at 17:24

2 Answers 2

The UCLA website has a bunch of great tutorials for every procedure broken down by the software type that you're familiar with. Check out Annotated SPSS Output: Logistic Regression -- the SES variable they mention is categorical (and not binary). SPSS will automatically create the indicator variables for you. There's also a page dedicated to Categorical Predictors in Regression with SPSS which has specific information on how to change the default codings and a page specific to Logistic Regression.

share|improve this answer

Logistic regression is a pretty flexible method. It can readily use as independent variables categorical variables. Most software that use Logistic regression should let you use categorical variables.

As an example, let's say one of your categorical variable is temperature defined into three categories: cold/mild/hot. As you suggest you could interpret that as three separate dummy variables each with a value of 1 or 0. But, the software should let you use a single categorical variable instead with text value cold/mild/hot. And, the logit regression would derive coefficient (or constant) for each of the three temperature conditions. If one is not significant, the software or the user could readily take it out (after observing t stat and p value).

The main benefit of grouping categorical variable categories into a single categorical variable is model efficiency. A single column in your model can handle as many categories as needed for a single categorical variable. If instead, you use a dummy variable for each categories of a categorical variable your model can quickly grow to have numerous columns that are superfluous given the mentioned alternative.

share|improve this answer
1  
@gaetan I do not understand the remark about a single column vs multiple columns. Are you suggesting that categorical variables should be coded as 1, 2, 3 etc in a single column instead of using dummy variables? I am not sure that makes sense to me as you are then imposing an implicit constraint that the difference in the effect on dv between leve1s 1 and 2 is the same as the difference in the effect on dv between levels 2 and 3. Perhaps, I am missing something. –  user28 Oct 7 '10 at 16:07
1  
@Gaetan I am not sure I follow you. How exactly does XLStat transform the 'text' values of cold, mild or hot into numerical values for the purpose of estimation? If there is a method that will let you estimate the effects of categorical variables without using dummy variables surely that should be independent of the software you use as there should be some underlying conceptual/model based logic. –  user28 Oct 7 '10 at 17:35
    
@Gaetan I don't follow your point unless you consider that your ordinal variable is treated as a continuous one (this might make sense sometimes, although we clearly assume that the variable can inherit the property of an interval scale as pointed by @Skrikant). Usually, a variable with $k$ levels is represented in the design matrix as $k-1$ columns, and I think this is quite independent of the software used (surely, XLStat takes care of constructing the correct design matrix as R, SPSS or Stata does). –  chl Oct 7 '10 at 18:42
1  
@Gatean Ok, in this case, the same can be done in SPSS (you have the choice between numerical/ordinal/nominal for each variable) -- then, the design matrix is constructed accordingly. –  chl Oct 7 '10 at 19:21
2  
@Gaetan @chl To summarize my understanding: The features of SPSS and XLStat whereby you can specify the measurement scale (nominal, ordinal etc) decreases the data file size. However, in both instances, the software uses the correct coding scheme (e.g., expand a nominal variable with J categories into J-1 dummy variables) as part of the estimation process in the background. Would that be a fair assessment of the situation? –  user28 Oct 7 '10 at 19:32

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.