Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
In the svm function, you can apply three cases to the kernel parameter. "Linear," "radial," and "polynomia."
And I try to derive the optimal svm result by adjusting cost, gamma and degree parameters.
What values should be used in each kernel?
What parameter do i have to use in each kernel????????
The Cost parameter is not a kernel parameter is an SVM parameter, that is why is common to all the three cases. The linear kernel does not have any parameters, the radial kernel uses the gamma parameter and the polynomial kernel uses the gamma, degree and also coef_0 (constant term in polynomial) parameters.
The choice of the kernel parameters and cost depends on your particular problem, to determine the best parameters you have to perform a model search over possible values, this can be computationally expensive (if you search over a large parameter space) and some times is not really necessary and you end up overfitting your model to your validation set. Most times you really just want a set of good enough parameters.
Speaking from experience a good starting point for SVMs is to use a linear kernel or a radial kernel with whatever the default gamma value is, and manually adjust the cost in powers of 10 (1e-3, 1e-2, 1e-1, 1, 10) to find the best value (evaluate this on a validation set). For most standard problems there is usually no visible benefit in using a polynomial kernel.
The reason using the default value of the gamma parameter in the radial kernel is that because of the mathematical formulation of the radial kernel you can usually get similar results by adjusting just the cost. The default gamma value for sklearn SVM for example is $\frac{1}{n_{features}}$.
