2
$\begingroup$

I have C and gamma parameters for RBF kernel to perform SVM classification through cross validation in R software. How to fix values for grid search to tune C and gamma parameters? For example I took grid ranging from [50 , 60 , 70 ....,600] for C and Gamma [ 0.05, 0.10,....,1]. I used a validation set for fine tuning the parameters. I fixed the gamma value and varied the C and got the optimum C value. Then I fixed the optimum C value and varied the gamma values to find the optimum gamma value. Is it right or are there any other way to perform effective grid search?

$\endgroup$
1
$\begingroup$

The C and gamma parameters influence each other, as you can see here:

C vs Gamma
(source: scikit-learn.org)

The performance of your solution depends on the initial fixed gamma value. If you choose a bad initial gamma value, you'll end up with a bad solution.

The easiest, but most time consuming way to find C and gamma is to test the whole grid of C x gamma values.

I often use some kind of (bayesian) optimization algorithm like this one (it's for Python, but similar should exist for R). It normally finds good C and gamma values in relatively few iterations.

PS.: the C and gamma values should be taken from a logarithmic grid of values like 10**[-5..5] - using a linear grid like [50, 60, ... ,600] won't work well.

$\endgroup$
0
$\begingroup$

in R you can do this by using tune.svm function of e1071 package for eg

obj = tune.svm(x,y,cost=10:100,gamma=seq(0,3,0.1))

would give you best cost and gamma value

please note that the values for cost and gamma are for understanding purpose only

$\endgroup$
1
$\begingroup$

Both C and gamma are scale parameters, so the grid should be on a logarithmic scale. I find that doubling/halving C and gamma on adjacent grid points is a good compromise (still working on the paper). For most problems the validation error will be non-convex, but generally unimodal, so gradient free optimisation tools, such as the Nelder-Mead simplex method are fairly efficient (I suspect someone may have implemented that for R, but I am a MATLAB user).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.