Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 4 down vote favorite

I have to do a binary logistic regression with a lot of independent variables. Most of them are binary. Few ones are categorical with more than two possible values.

Which is the best way to deal with such variables?

For example, for a variable with three possible values, I suppose that two dummy variables have to be created. Then, in a step-wise regression procedure, it is better to test in the same time both the two dummy variables, or to test them separately?

Since I will use SPSS which I do not remember very well, which way does SPSS operate?

Moreover, for a categorical ordinal variable, it is a good thing to use dummy variables which recreate the ordinal scale (for example, using three dummy variables for a 4-state ordinal variable, put 0-0-0 for level 1, 1-0-0 for level 2, 1-1-0 for level 3 and 1-1-1 for level 4, instead of 0-0-0, 1-0-0, 0-1-0 and 0-0-1 for the 4 levels)?

Thanks for your answers.

share|improve this question
2  
This is just a partial answer: even when you create the dummies explicitly (rather than using the software's implicit capabilities), keep them together in all analyses. In particular, they should all enter together and all leave together in a stepwise regression, with the p-value computed appropriately for the total number of variables involved. (This is Hosmer & Lemeshow's recommendation, anyway, and it makes a lot of sense.) – whuber♦ Oct 7 '10 at 20:53
I wrote a post a while back on multinomial logistic regression resources in SPSS. – Jeromy Anglim Oct 8 '10 at 5:44
You're talking about your independent variables. It's only the dependent variables that need to be binary for logistic regression. – John Aug 30 '12 at 13:43

2 Answers

up vote 6 down vote

The UCLA website has a bunch of great tutorials for every procedure broken down by the software type that you're familiar with. Check out Annotated SPSS Output: Logistic Regression -- the SES variable they mention is categorical (and not binary). SPSS will automatically create the indicator variables for you. There's also a page dedicated to Categorical Predictors in Regression with SPSS which has specific information on how to change the default codings and a page specific to Logistic Regression.

share|improve this answer
up vote 1 down vote

Logistic regression is a pretty flexible method. It can readily use as independent variables categorical variables. Most software that use Logistic regression should let you use categorical variables.

As an example, let's say one of your categorical variable is temperature defined into three categories: cold/mild/hot. As you suggest you could interpret that as three separate dummy variables each with a value of 1 or 0. But, the software should let you use a single categorical variable instead with text value cold/mild/hot. And, the logit regression would derive coefficient (or constant) for each of the three temperature conditions. If one is not significant, the software or the user could readily take it out (after observing t stat and p value).

The main benefit of grouping categorical variable categories into a single categorical variable is model efficiency. A single column in your model can handle as many categories as needed for a single categorical variable. If instead, you use a dummy variable for each categories of a categorical variable your model can quickly grow to have numerous columns that are superfluous given the mentioned alternative.

share|improve this answer
1  
@gaetan I do not understand the remark about a single column vs multiple columns. Are you suggesting that categorical variables should be coded as 1, 2, 3 etc in a single column instead of using dummy variables? I am not sure that makes sense to me as you are then imposing an implicit constraint that the difference in the effect on dv between leve1s 1 and 2 is the same as the difference in the effect on dv between levels 2 and 3. Perhaps, I am missing something. – user28 Oct 7 '10 at 16:07
1  
@Gaetan I am not sure I follow you. How exactly does XLStat transform the 'text' values of cold, mild or hot into numerical values for the purpose of estimation? If there is a method that will let you estimate the effects of categorical variables without using dummy variables surely that should be independent of the software you use as there should be some underlying conceptual/model based logic. – user28 Oct 7 '10 at 17:35
@Gaetan I don't follow your point unless you consider that your ordinal variable is treated as a continuous one (this might make sense sometimes, although we clearly assume that the variable can inherit the property of an interval scale as pointed by @Skrikant). Usually, a variable with $k$ levels is represented in the design matrix as $k-1$ columns, and I think this is quite independent of the software used (surely, XLStat takes care of constructing the correct design matrix as R, SPSS or Stata does). – chl♦ Oct 7 '10 at 18:42
1  
@Gatean Ok, in this case, the same can be done in SPSS (you have the choice between numerical/ordinal/nominal for each variable) -- then, the design matrix is constructed accordingly. – chl♦ Oct 7 '10 at 19:21
2  
@Gaetan @chl To summarize my understanding: The features of SPSS and XLStat whereby you can specify the measurement scale (nominal, ordinal etc) decreases the data file size. However, in both instances, the software uses the correct coding scheme (e.g., expand a nominal variable with J categories into J-1 dummy variables) as part of the estimation process in the background. Would that be a fair assessment of the situation? – user28 Oct 7 '10 at 19:32
show 3 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.