Lets say I have a highly dimensional classification problem with a lot of noise, and I want to improve my results by removing some of the noisy variables. I've read several papers on using SVMs for feature selection, but I'm at a loss as to how to implement this in R. Are there pre-existing packages that do this, or am I going to have to roll my own?
Tell me more
×
Cross Validated is a question and answer site for
statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
|
As I understand them, SVMs have built-in regularization because they tend to penalize large weights of predictors which amounts to favor simpler models. They're often used with recursive feature elimination (in neuroimaging paradigms, at least). About R specifically, there's the kernlab package, by Alex Smola who co-authored Learning with Kernels (2002, MIT Press), which implements SVM (in addition to e1071). However, if you are after a dedicated framework, I would warmly recommend the caret package. |
|||||||||
|
|
For Recursive Feature Extraction (SVM-RFE) the packages e1071 and Kernlab doesn't implement it i think. For the Weka SVMAttributeEval package is for Java i think, but the question was for R as i saw. The best way is trying to implement the SVM-RFE using e1071 and LIBSVM library I found a good parper relating that here. |
|||
|
|
|
The Weka SVMAttributeEval package allows you to do feature selection using SVM. It should be pretty easy to dump your R data frame to a csv file, import that into Weka, do the feature selection, and then pull it back into R. |
|||
|
|
