7
$\begingroup$

I am having trouble determining what kernel I should use in a non-linear SVM without testing in advance. I want to know if there are any other ways to determine the best kernel without tests? How does it relate to the data?

$\endgroup$
14
$\begingroup$

Do your analysis with several different kernels. Make sure you cross-validate. Choose the kernel that performs the best during cross-validation and fit it to your whole dataset.

/edit: Here is some example code in R, for a classification SVM:

#Use a support vector machine to predict iris species
library(caret)
library(caTools)

#Choose x and y
x <- iris[,c("Sepal.Length","Sepal.Width","Petal.Length","Petal.Width")]
y <- iris$Species

#Pre-Compute CV folds so we can use the same ones for all models
CV_Folds <- createMultiFolds(y, k = 10, times = 5)

#Fit a Linear SVM
L_model <- train(x,y,method="svmLinear",tuneLength=5,
    trControl=trainControl(method='repeatedCV',index=CV_Folds))

#Fit a Poly SVM
P_model <- train(x,y,method="svmPoly",tuneLength=5,
    trControl=trainControl(method='repeatedCV',index=CV_Folds))

#Fit a Radial SVM
R_model <- train(x,y,method="svmRadial",tuneLength=5,
    trControl=trainControl(method='repeatedCV',index=CV_Folds))

#Compare 3 models:
resamps <- resamples(list(Linear = L_model, Poly = P_model, Radial = R_model))
summary(resamps)
bwplot(resamps, metric = "Accuracy")
densityplot(resamps, metric = "Accuracy")

#Test a model's predictive accuracy Using Area under the ROC curve
#Ideally, this should be done with a SEPERATE test set
pSpecies <- predict(L_model,x,type='prob')
colAUC(pSpecies,y,plot=TRUE)
$\endgroup$
  • $\begingroup$ Is the cross validation giving me the best frontier fitting my dataset ? I mean there not different parameters to decide the best kernels ? $\endgroup$ – 404Dreamer_ML May 9 '11 at 14:26
  • 1
    $\begingroup$ Each will also need some tuning to select the best parameters for that kernel. This tuning should also be done via cross-validation, WITHIN the overall cross-validation. SVMs are tricky. $\endgroup$ – Zach May 9 '11 at 15:30
  • $\begingroup$ Thx :) Very Helpful $\endgroup$ – 404Dreamer_ML May 9 '11 at 15:39
  • 1
    $\begingroup$ @404Dreamer_ML I added some example code. There are other, better ways to do this (i.e. use a separate test set to compare your models after you've fit them, or better yet cross-validation), but this code will at least give you a framework to work from. $\endgroup$ – Zach May 9 '11 at 15:57
  • $\begingroup$ For your linear SVM model, the tuneLength parameter is redundant, the default for caret is always to use a cost parameter of 1 no matter what tuneLength. L_model <- train(x,y,method="svmLinear",tuneLength=5, trControl=trainControl(method='repeatedCV',index=CV_Folds)) To vary the C parameter you need to define a tuneGrid. See: github.com/topepo/caret/issues/636 $\endgroup$ – samleighton87 Oct 15 '17 at 19:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.