3
$\begingroup$

can anyone direct me to a package/commands in R for performing step-wise feature selection, preferably using the caret package.

I have already used linear discriminant analysis (LDA), Random forest, PCA and a wrapper using a support vector machine. I was thinking of including a partial least sqaures or a gradient boosting method, but while trying to use them on multi-class data, they cause R to crash. People have reported similar experiences on multi-class data using caret when attempting to use gbm.

I realize that I haven't used a step-wise approach and I was searching for one that can be implemented on highly correlated, dependent variables for selecting the best performing 20 variables (for example) to create a parsimonious model.

Any suggestions would be welcomed

Improve this question
$\endgroup$

1 Answer 1

Reset to default
8
$\begingroup$

caret has a stepLDA method available in train:

slda <- train(Species ~ ., data = iris,
              method = "stepLDA",
              trControl = trainControl(method = "cv"))

This uses stepclass in the klaR package. There are also LDA feature selection tools in caret using rfe and sbf that would be helpful.

Max

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks Max, wondering if you have gotten around to solving the issue with gbm, related to the post here? Thanks though for your suggestion. $\endgroup$
    – user2507608
    Nov 23, 2013 at 5:06
  • $\begingroup$ I think so. Try updating caret and gbm. $\endgroup$
    – topepo
    Nov 26, 2013 at 16:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.