0
$\begingroup$

I have a classification problem with a modest number of records (approx. 10,000) and dimensions (30 dimensions, 25 are categoric and 5 are numeric). The response variable has two classes (T/F).

I'm using rfe() from the caret package to do feature selection. Here's my code-

ctrl <- rfeControl(functions = rfFuncs,
                      method = "repeatedcv",
                      repeats = 3,
                      verbose = FALSE)
results2 <- rfe(trainSet[,predictors], 
                as.factor(trainSet[,outcomeName]), 
                sizes=c(1:100), 
                rfeControl=ctrl)

Many of the categoric variables are correlated with one another. I was curious how that affects the results that come from this wrapper method of feature selection.

I know in the context of regression problems, multicollinearity can pose a large problem because the model makes a somewhat arbitrary selection when determining the coefficients with the related variables.

Does rfe() do a better job with correlated variables? Is this a form of data drudging, since it makes a somewhat arbitrary selection for the one that creates a better classification accuracy?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.