In answer to a previous question factor pooling in model selection was discussed.

If a factor or categorical variable is to be dropped in model selection, should all levels be dropped simultaneously? If so, why?

The motivation for dropping factors is to aid model interpretation. For example, I might be interested in explaining the factors that influence customer behaviour when visiting a store and have a categorical variable "travel mode" with factors "walking, bus, private car, taxi, etc." In this context, I can remove all the dummy variables except "private car" because they have an insubstantial estimated magnitude and are not significant predictors of behaviour. I then end up with a "travelled in private car" vs "didn't travel in private car" variable and don't have to worry about troubling the reader with interpreting the other largely uninteresting variables.

  • 1
    $\begingroup$ What scientific sense would it make to retain some, but not all, levels of a factor? How would you interpret that? $\endgroup$ – whuber Nov 22 '11 at 3:48
  • 2
    $\begingroup$ +1 @fmark I'm glad you started a new question. I think this is the right format for your questions to be discussed to your satisfaction. Comments just aren't enough. $\endgroup$ – gung - Reinstate Monica Nov 22 '11 at 6:49
  • $\begingroup$ @whuber I think what fmark may be thinking of is something like multiple comparison tests after a significant ANOVA; when you run orthogonal contrasts, you are essentially collapsing levels together. $\endgroup$ – gung - Reinstate Monica Nov 22 '11 at 6:54
  • 1
    $\begingroup$ @rolando2 I don't think there was anything to strike me as nonsensical or uninterpretable: I was just asking. In some contexts dropping some levels of a factor makes no scientific sense and in others it can, as you indicate. Asking about interpretability is one way to approach fmark's question. $\endgroup$ – whuber Nov 23 '11 at 3:23
  • 1
    $\begingroup$ @whuber I've edited the question to explain why I might want to do this. $\endgroup$ – fmark Nov 23 '11 at 3:36

I'm really not sure what the answer would be in the absence of crossvalidation. But if we are crossvalidating, and we find that, say, one ethnic group out of 6 is substantially different from the others wrt Y, I can't seem to see anything wrong with using only that group's dummy variable in the followup equation. If membership/nonmembership in that group, and none other, is helping to predict the outcome (or to explain it, for that matter), why gummy up the equation with a bunch of unhelpful predictor dummies, which would only figure to add noise to the prediction?

| cite | improve this answer | |
  • 3
    $\begingroup$ Because removing insignificant terms only seems to help the predictions. It actually makes predictions worse in new data, and results in multiplicity problems that hurts type I error and confidence interval coverage. $\endgroup$ – Frank Harrell Nov 22 '11 at 20:53
  • $\begingroup$ This conversation has been picked up again at stats.stackexchange.com/questions/24298/… $\endgroup$ – rolando2 Mar 8 '12 at 20:21

Not the answer you're looking for? Browse other questions tagged or ask your own question.