0
$\begingroup$

Can anyone guide me about the feature selection based on correlation using SVM? RBF kernel check the correlation too or not? I am using weka and matlab. Any help would be appreciated.

$\endgroup$
  • $\begingroup$ Show us what have you done so far. Also, what do you mean feature selection based on correlation? $\endgroup$ – user2974951 Dec 19 '18 at 12:32
  • $\begingroup$ i have read a paper and they used SVM for feature selection based on correlation and i was wondering is that even posiible $\endgroup$ – snape Dec 19 '18 at 14:05
  • $\begingroup$ I don't use Weka or MATLAB so I can't help you there. But it's true feature selection wrapper methods exist for SVM, check stackoverflow.com/questions/17529537/… and kaggle.com/c/overfitting/discussion/487 $\endgroup$ – user2974951 Dec 20 '18 at 7:13
1
$\begingroup$

I would recommend not performing feature selection with SVMs if you are interested in generalisation performance. The SVM approximately implements a bound on the generalisation that is independent of the number of features. However, as soon as you try to optimise the kernel (or equivalently select features), then all of the theory underpinning the bound is violated and the bound no longer holds. It is easy to over-fit feature selection criteria, and this can make performance much worse. See, for example, my answer here.

If you are interested in finding the most important features as an end in itself (rather than to improve performance) then I would recommend using a model that has feature selection built in, such as the LASSO or LARS.

I can strongly recommend the paper of Guyon and Elisseeff for guidance on the use of feature selection (and some of the potential pitfalls).

$\endgroup$
0
$\begingroup$

No, RBF kernel doesn't check the correlation. You can need to find the out correlation separately and filter out the ones which are highly correlated and then build an SVM model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.