I'm using libsvm and the 3-fold cross validation to select the best C and gamma, but I'm not sure for the range to use in the grid search. Is there any standard way to choose this range? I used: log2c =(-10:3:4) and log2g=exp(-10:3:4)
If I change the tolerance -e, the result change, can any one clarify tis point for me?
thank you.
I don't think there is a standard way to choose a range (there are some heuristics for choosing a gamma value). I would look at the libsvm practical tutorial on getting started with SVMs. http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf
Generally, I don't think you usually see C values that are so low. I would start with 2^-3 to 2^10 for C values and keep the gamma range the same. I would then run a coarse to fine grid search. It looks like you're using steps of 4? I would then search the area around the most accurate parameter set at a smaller step size, like 1.
Hope this helps.
There isn't a standard range, if in doubt, generate a contour plot of the cross-validation criterion as a function of the hyper-parameters and see if the minimum is near one of the edges of the plot, and if this is the case, extend the grid-search in that direction.
Note that if the problem is (near) linear, IIRC the "optimal" kernel parameter value tends to minus infinity in an attempt to make the smoothest kernel possible. So if that happens you may want to try a linear SVM as well.
As @danjeharry (+1) suggests, the grid-search you are using at the moment is a bit coarse and you may get better results using a finer resolution grid search in the vicinity of the coarse grid search you have already performed. I would advise against making the step size less than say 1/2 as this can lead to over-fitting the model selection criterion and performance getting worse rather than better.
