8
$\begingroup$

I am having trouble determining what kernel I should use in a non-linear SVM without testing in advance. I want to know if there are any other ways to determine the best kernel without tests? How does it relate to the data?

$\endgroup$
1
  • $\begingroup$ What do people in your domain usually do? You might want to try to use the same kernel as a starting point; same with C and sigma/gamma values. $\endgroup$
    – GuillaumeL
    Jan 27, 2021 at 19:45

1 Answer 1

17
$\begingroup$

Do your analysis with several different kernels. Make sure you cross-validate. Choose the kernel that performs the best during cross-validation and fit it to your whole dataset.

/edit: Here is some example code in R, for a classification SVM:

#Use a support vector machine to predict iris species
library(caret)
library(caTools)

#Choose x and y
x <- iris[,c("Sepal.Length","Sepal.Width","Petal.Length","Petal.Width")]
y <- iris$Species

#Pre-Compute CV folds so we can use the same ones for all models
CV_Folds <- createMultiFolds(y, k = 10, times = 5)

#Fit a Linear SVM
L_model <- train(x,y,method="svmLinear",tuneLength=5,
    trControl=trainControl(method='repeatedCV',index=CV_Folds))

#Fit a Poly SVM
P_model <- train(x,y,method="svmPoly",tuneLength=5,
    trControl=trainControl(method='repeatedCV',index=CV_Folds))

#Fit a Radial SVM
R_model <- train(x,y,method="svmRadial",tuneLength=5,
    trControl=trainControl(method='repeatedCV',index=CV_Folds))

#Compare 3 models:
resamps <- resamples(list(Linear = L_model, Poly = P_model, Radial = R_model))
summary(resamps)
bwplot(resamps, metric = "Accuracy")
densityplot(resamps, metric = "Accuracy")

#Test a model's predictive accuracy Using Area under the ROC curve
#Ideally, this should be done with a SEPERATE test set
pSpecies <- predict(L_model,x,type='prob')
colAUC(pSpecies,y,plot=TRUE)
$\endgroup$
5
  • $\begingroup$ Is the cross validation giving me the best frontier fitting my dataset ? I mean there not different parameters to decide the best kernels ? $\endgroup$ May 9, 2011 at 14:26
  • 1
    $\begingroup$ Each will also need some tuning to select the best parameters for that kernel. This tuning should also be done via cross-validation, WITHIN the overall cross-validation. SVMs are tricky. $\endgroup$
    – Zach
    May 9, 2011 at 15:30
  • $\begingroup$ Thx :) Very Helpful $\endgroup$ May 9, 2011 at 15:39
  • 1
    $\begingroup$ @404Dreamer_ML I added some example code. There are other, better ways to do this (i.e. use a separate test set to compare your models after you've fit them, or better yet cross-validation), but this code will at least give you a framework to work from. $\endgroup$
    – Zach
    May 9, 2011 at 15:57
  • $\begingroup$ For your linear SVM model, the tuneLength parameter is redundant, the default for caret is always to use a cost parameter of 1 no matter what tuneLength. L_model <- train(x,y,method="svmLinear",tuneLength=5, trControl=trainControl(method='repeatedCV',index=CV_Folds)) To vary the C parameter you need to define a tuneGrid. See: github.com/topepo/caret/issues/636 $\endgroup$ Oct 15, 2017 at 19:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.