I have a classification problem with a modest number of records (approx. 10,000) and dimensions (30 dimensions, 25 are categoric and 5 are numeric). The response variable has two classes (T/F).
rfe() from the caret package to do feature selection. Here's my code-
ctrl <- rfeControl(functions = rfFuncs, method = "repeatedcv", repeats = 3, verbose = FALSE) results2 <- rfe(trainSet[,predictors], as.factor(trainSet[,outcomeName]), sizes=c(1:100), rfeControl=ctrl)
Many of the categoric variables are correlated with one another. I was curious how that affects the results that come from this wrapper method of feature selection.
I know in the context of regression problems, multicollinearity can pose a large problem because the model makes a somewhat arbitrary selection when determining the coefficients with the related variables.
rfe() do a better job with correlated variables? Is this a form of data drudging, since it makes a somewhat arbitrary selection for the one that creates a better classification accuracy?