Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.

I am having a big trouble to determine the kernel i should use in a non linear SVM without testing before, I want to know if there is any other ways to determine the best kernel without tests ? Is it linked to the data we are working on ?

share|improve this question

1 Answer

up vote 5 down vote accepted

Do your analysis with several different kernels. Make sure you cross-validate. Choose the kernel that performs the best during cross-validation and fit it to your whole dataset.

/edit: Here is some example code in R, for a classification SVM:

#Use a support vector machine to predict iris species
library(caret)
library(caTools)

#Choose x and y
x <- iris[,c("Sepal.Length","Sepal.Width","Petal.Length","Petal.Width")]
y <- iris$Species

#Pre-Compute CV folds so we can use the same ones for all models
CV_Folds <- createMultiFolds(y, k = 10, times = 5)

#Fit a Linear SVM
L_model <- train(x,y,method="svmLinear",tuneLength=5,
    trControl=trainControl(method='repeatedCV',index=CV_Folds))

#Fit a Poly SVM
P_model <- train(x,y,method="svmPoly",tuneLength=5,
    trControl=trainControl(method='repeatedCV',index=CV_Folds))

#Fit a Radial SVM
R_model <- train(x,y,method="svmRadial",tuneLength=5,
    trControl=trainControl(method='repeatedCV',index=CV_Folds))

#Compare 3 models:
resamps <- resamples(list(Linear = L_model, Poly = P_model, Radial = R_model))
summary(resamps)
bwplot(resamps, metric = "Accuracy")
densityplot(resamps, metric = "Accuracy")

#Test a model's predictive accuracy Using Area under the ROC curve
#Ideally, this should be done with a SEPERATE test set
pSpecies <- predict(L_model,x,type='prob')
colAUC(pSpecies,y,plot=TRUE)
share|improve this answer
Is the cross validation giving me the best frontier fitting my dataset ? I mean there not different parameters to decide the best kernels ? – 404Dreamer_ML May 9 '11 at 14:26
Each will also need some tuning to select the best parameters for that kernel. This tuning should also be done via cross-validation, WITHIN the overall cross-validation. SVMs are tricky. – Zach May 9 '11 at 15:30
Thx :) Very Helpful – 404Dreamer_ML May 9 '11 at 15:39
1  
@404Dreamer_ML I added some example code. There are other, better ways to do this (i.e. use a separate test set to compare your models after you've fit them, or better yet cross-validation), but this code will at least give you a framework to work from. – Zach May 9 '11 at 15:57

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.