2
$\begingroup$

So I have some experience when it comes to SVM mainly through a basic course in machine learning. However when it comes to features I've never needed to do any form of feature selection before.

The dataset that I have right now has around 40 features and about 650 data points. I've read about different feature selection techniques and decided to try out recursive feature elimination, RFE.

I've tried using the caret package for R and scikit for python 3 however I'm rather confused with the kernels used. It would seem that RFE is mostly done with a linear kernel so my question becomes if you can perform RFE with other kernels like radial, sigmoid and poly.?

$\endgroup$
2
$\begingroup$

You can read what the authors have to say about your question here. They wrote:

The method of eliminating features on the basis of the smallest change in cost function described in Section [...] can be extended to the non-linear case and to all kernel methods in general (Weston, 2000(b)). One can make computations tractable by assuming no change in the value of the α’s. Thus one avoids having to retrain a classifier for every candidate feature to be eliminated

So i guess is possible, but not implemented in scikit and R.

$\endgroup$
  • $\begingroup$ No problem. I'd be interested if you find/implement a solution for this. $\endgroup$ – hh32 Nov 8 '16 at 15:13
  • $\begingroup$ I found a paper on it, researchgate.net/publication/…, I'll try to use it. $\endgroup$ – user5232061 Nov 8 '16 at 18:17
0
$\begingroup$

Yes, you can. It's not hard to implement. if you decide to use caret, look up train_model_list in caret(https://cran.r-project.org/web/packages/caret/caret.pdf) P164 - A List of Available Models in train, this section describes methods you can use such as Distance Weighted Discrimination with Polynomial Kernel (method = 'dwdPoly') and Distance Weighted Discrimination with Radial Basis Function Kernel (method = 'dwdRadial').

I also came across msvm-rfe package,http://www.colbyimaging.com/wiki/statistics/msvm-rfe, the example showed using RBF kernel doing RFE.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.