$\nu$-svm parameter selection

For the $\nu$-SVM (for both classification and regression cases) the $\nu \in (0;1)$ should be selected. The LIBSVM guide suggests to use grid search for identifying the optimal value of the $C$ parameter for $C$-SVM, it also recommends to try following values $C = 2^{-5}, 2^{-3}, \dots, 2^{15}$.

So the question is, are there any recommendations for values of the $\nu$ parameter in case of $\nu$-SVMs?

-

Rather than use a grid search, you can just optimise the hyper-parameters using standard numeric optimisation techniques (e.g. gradient descent). If you don't have estimates of the gradients, you can use the Nelder-Mead simplex method, which doesn't require gradient information and is vastly more efficient than grid-search methods. I would use the logit function to map the (0;1) range of $\nu$ onto $(-\infty;+\infty)$ to get an unconstrained optimisation problem.

If you really want to use grid search, then just spacing the evaluation points linearly in the range 0 - 1 should be fine.

-
Let me clarify your answer. So you suggest to use gradient descent like methods for SVM loss function minimization (that depends on $\nu, \alpha_1, \dots, \alpha_n$), instead of SMO method that optimizes loss function for fixed $\nu$. Right? To use gradient descent, the loss function should be unimodal. So does the loss function unimodal with respect to $\nu$ variable? –  TheBug Apr 7 '11 at 8:29
@TheBug, no, the SVM itself is still trained using SMO, but the hyper-parameters ($\nu$ and any kernel parameters) are optimised by gradient descent (or Nelder Mead simplex) minimisation of some suitable model selection criterion (e.g. cross-validation error). At each step you have to train several SVMs with fixed $\nu$ in order to define the simplex or to compute the gradient (using finite differences if it can't be done analytically). The cross-validation error is not generally unimodal, but it is normally smooth and such methods work well in practice. –  Dikran Marsupial Apr 7 '11 at 9:17
Thank you, now it's clear. –  TheBug Apr 7 '11 at 9:58
One more question about this technique. Can be same method applied to find $C$ and $\nu$ for $\nu$-SV regression? –  TheBug Apr 23 '11 at 12:56
It should be O.K., I haven't used $\nu$-SVR, but I regularly use these sorts of methods for optimising the hyper-parameters of a wide range of kernel models and Gaussian processes (both regression and classification). As long as you apply a suitable transformation to get an unconstrained optimisation problem, it seems pretty reliable. –  Dikran Marsupial Apr 23 '11 at 13:00