# Grid search and tolerance in libsvm

1. I'm using libsvm and the 3-fold cross validation to select the best C and gamma, but I'm not sure for the range to use in the grid search. Is there any standard way to choose this range? I used: log2c =(-10:3:4) and log2g=exp(-10:3:4)

2. If I change the tolerance -e, the result change, can any one clarify tis point for me?

thank you.

-
Welcome to the site! For the future -- please rather ask separate questions for separate problems. –  mbq Feb 28 '12 at 15:18

I don't think there is a standard way to choose a range (there are some heuristics for choosing a gamma value). I would look at the libsvm practical tutorial on getting started with SVMs. http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf

Generally, I don't think you usually see C values that are so low. I would start with 2^-3 to 2^10 for C values and keep the gamma range the same. I would then run a coarse to fine grid search. It looks like you're using steps of 4? I would then search the area around the most accurate parameter set at a smaller step size, like 1.

Hope this helps.

-
thank you for you reply it is usfull, –  libsvm Mar 1 '12 at 19:35
how about the tolerance -e ? what is it effect? –  libsvm Mar 1 '12 at 19:46

There isn't a standard range, if in doubt, generate a contour plot of the cross-validation criterion as a function of the hyper-parameters and see if the minimum is near one of the edges of the plot, and if this is the case, extend the grid-search in that direction.

Note that if the problem is (near) linear, IIRC the "optimal" kernel parameter value tends to minus infinity in an attempt to make the smoothest kernel possible. So if that happens you may want to try a linear SVM as well.

As @danjeharry (+1) suggests, the grid-search you are using at the moment is a bit coarse and you may get better results using a finer resolution grid search in the vicinity of the coarse grid search you have already performed. I would advise against making the step size less than say 1/2 as this can lead to over-fitting the model selection criterion and performance getting worse rather than better.

-