I have a classification problem with a modest number of records (approx. 10,000) and dimensions (30 dimensions, 25 are categoric and 5 are numeric). The response variable has two classes (T/F).

I'm using rfe() from the caret package to do feature selection. Here's my code-

ctrl <- rfeControl(functions = rfFuncs,
                      method = "repeatedcv",
                      repeats = 3,
                      verbose = FALSE)
results2 <- rfe(trainSet[,predictors], 

Many of the categoric variables are correlated with one another. I was curious how that affects the results that come from this wrapper method of feature selection.

I know in the context of regression problems, multicollinearity can pose a large problem because the model makes a somewhat arbitrary selection when determining the coefficients with the related variables.

Does rfe() do a better job with correlated variables? Is this a form of data drudging, since it makes a somewhat arbitrary selection for the one that creates a better classification accuracy?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.