Before training a glm model (in R), predictors were transformed into matrix and highly correlated/near zero variance variables were excluded:

x<-training[,c(3:25, 27:39)]
x2<-model.matrix(~., data = x)[,-1]
nzv <- nearZeroVar(x2,freqCut = 100/1)
x3 <- x2[, -nzv]
corr_mat <- cor(x3)
too_high <- findCorrelation(corr_mat, cutoff = .9)
x4 <- x3[, -too_high]

My questions is, since categorical variables with multiple levels were transformed into dummy variables, if this procedure excludes some of the dummy variables, how do we interpret the estimated results for the rest of the dummy variables? For example, categorical variable X has four levels (A,B,C,D) that we transform into three dummy variables (A,B,C). Variable selection procedure excludes variable A, then we run the model with variable B and C, how do we interpret the coefficients for B&C?

This situation may also happen when conducting variable selection using the rfe function in the caret package as well. Any suggestion as to how to incorporate what Peter suggested below (exclude all A, B,C if one of them is excluded) into the rfe function? This second question has been moved to Stackoverflow. Please go to the following link if you know the answer or have the same question:


Many thanks!

  • $\begingroup$ Do we interpret the coefficients for B&C the same as if we did not drop A? $\endgroup$ – ybeybe May 18 '16 at 21:57

Do not allow variable selection to exclude A unless it also excludes B and C.

There is probably some interpretation of a model with only A deleted, but it doesn't seem to be anything that makes sense.

  • $\begingroup$ Thank you Peter. I agree that the interpretation would not make much sense...I can certainly try excluding the whole variable if one of the levels if identified as nzv or highly correlated with other variables. $\endgroup$ – ybeybe May 19 '16 at 15:54
  • 1
    $\begingroup$ Similar challenges when using the rfe function in the caret package to select variable (variable selection using cross validation) - dummy variables may be dropped during the process. Any suggestion regarding this function in particular? $\endgroup$ – ybeybe May 19 '16 at 15:56
  • $\begingroup$ Peter, I feel my follow up question has moved from stats to programming in R. I'll move on to Stack Overflow. Thanks again. $\endgroup$ – ybeybe May 19 '16 at 16:14
  • $\begingroup$ Sure. After using whatever you are using, re-run the regression with the other dummy variables included. $\endgroup$ – Peter Flom - Reinstate Monica May 19 '16 at 17:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.