0
$\begingroup$

I have developed a support vector machine model with 78 features on which I would like to apply recursive feature elimination. I developed the model with the package ‘rminer’.

fitmodel<-fit(Sustainable.Product.Choice ~ ., data = Sus.Food.Train.Smote, model = "svm", task = "class", search = "grid")

The model uses an RBF kernel. It yields satisfactory results, however, I was wondering if every feature would necessarily need to be kept in the model and if I could improve the classification results (and generalizability) by excluding some of the features. I therefore tried to apply recursive feature elimination with the package ‘caret’ using the following code, but this did not return any result (R just keeps on processing indefinitely…):

x<-as.data.frame(Sus.Food.Train.Smote[,1:78])

y<-as.factor(Sus.Food.Train.Smote[,79])

ctrl<-rfeControl(functions=caretFuncs, method="repeatedcv", repeats=5, verbose=FALSE)

REFEL<-rfe(x, y, rfeControl=ctrl, method="svmRadial")

I also tried out different values for the ‘sizes’ argument, but that did not change the outcome.

A second question relates to the sequence of the procedure: Should I either

  1. apply recursive feature elimination before I develop the actual support vector model?

  2. develop the support vector model first, assess its quality, and afterwards apply recursive feature elimination before running the support vector model again with the reduced set of features? In this case, how can I tell R that the feature elimination should rely on the specific support vector model I developed beforehand?

Thank you very much in advance.

Kind regards,

Hannes

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.