Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 6 down vote favorite

I'm doing a simple AIC-based backward elimination model where some variables are categorical variables with multiple levels. These variables are modeled as a set of dummy variables. When doing backward elimination, should I be removing all the levels of a variable together? Or should I treat each dummy variable separately? And why?

As a related question, step in R handles each dummy variable separately when doing backward elimination. If I wanted to remove an entire categorical variable at once, can I do that using step? Or are there alternatives to step which can handle this?

Thanks in advance.

share|improve this question
I think it is hard to answer this question in a general context. If possible, can you pls provide some insights about what is your data? what do you want to infer? possible assumptions, etc. Providing these details might help us in answering your questions better. – suncoolsu Jan 7 '11 at 1:02
@suncoolsu I'm not really at liberty to discuss any of that. If it's possible to answer the question with a decision tree ('if you're assuming X, then do Y'), that could be really helpful. – nerdbound Jan 7 '11 at 2:11
1  
For clarity, are you interested in interpreting the model as causal or are you simply interested in prediction? – Andy W Jan 7 '11 at 2:18

1 Answer

up vote 5 down vote accepted

I think you'd have to remove the entire categorical variable. Imagine a logistic regression in which you're trying to predict if a person has a disease or not. Country of birth might have a major impact on that, so you include it in your model. If the specific USAmerican origin didn't have any impact on AIC and you dropped it, how would you calculate $\hat{y}$ for an American? R uses reference contrasts for factors by default, so I think they'd just be calculated at the reference level (say, Botswana), if at all. That's probably not going to end well...

A better option would be to sort out sensible encodings of country of birth beforehand - collapsing into region, continent, etc. and finding which of those is most suitable for your model.

Of course, there are many ways to misuse stepwise variable selection, so make sure that you're doing it properly. There's plenty about that on this site, though; searching for "stepwise" brings up some good results. This is particularly pertinent, with lots of good advice in the answers.

share|improve this answer
1  
Thanks for the response. I guess my concern is that, in your example, if one country of birth is a fantastic predictor of a disease, and all the other countries of origin are poor predictors, then the backward elimination model will tell me to remove country of birth, even though country of birth is hardly useless as a covariate. In other words, it might make sense to have a single variable for 'are you Lithuanian?' even though country of birth as a whole does not predict well. Are there sensible ways to handle that sort of problem? – nerdbound Jan 7 '11 at 1:53
2  
(+1) If one country is a great predictor, then the whole group of countries will be significant and it will be retained. @Matt Parker is correct: keep groups of dummy variables together and base the entering and leaving criteria on the joint significance level for the entire group, not on the significance levels for the individual dummies. – whuber♦ Jan 7 '11 at 2:18
1  
@whuber OK one more potentially dumb comment from me and then I'll be at peace when I understand why I'm wrong :-P It seems to me that if I'm looking at AIC and I exclude, say, 200 country dummies as a group, the AIC will improve because the new model will have 200 fewer variables, and it will get worse only to the extent that those dummies were good predictors. If only one of the variables was a good predictor, it seems to me that on net the AIC will make the model with 200 fewer variables look better, but then I'll be missing out on that one variable... – nerdbound Jan 7 '11 at 3:03
2  
@nerdbound That's not dumb at all. However, if only one dummy in 200 works well, you really have a different categorical variable: it's a new binary (or reclassified) variable. @Matt Parker addresses this issue in his second paragraph. This looks like a matter for exploratory analysis (which is what stepwise regression is anyway). It's perfectly fine to torture your data in this way, but after all the smoke clears, you need to test your model on independent data. – whuber♦ Jan 7 '11 at 3:09

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.