Before training a glm model (in R), predictors were transformed into matrix and highly correlated/near zero variance variables were excluded:
x<-training[,c(3:25, 27:39)] x2<-model.matrix(~., data = x)[,-1] nzv <- nearZeroVar(x2,freqCut = 100/1) x3 <- x2[, -nzv] corr_mat <- cor(x3) too_high <- findCorrelation(corr_mat, cutoff = .9) x4 <- x3[, -too_high]
My questions is, since categorical variables with multiple levels were transformed into dummy variables, if this procedure excludes some of the dummy variables, how do we interpret the estimated results for the rest of the dummy variables? For example, categorical variable X has four levels (A,B,C,D) that we transform into three dummy variables (A,B,C). Variable selection procedure excludes variable A, then we run the model with variable B and C, how do we interpret the coefficients for B&C?
This situation may also happen when conducting variable selection using the rfe function in the caret package as well. Any suggestion as to how to incorporate what Peter suggested below (exclude all A, B,C if one of them is excluded) into the rfe function? This second question has been moved to Stackoverflow. Please go to the following link if you know the answer or have the same question: