I am trying to do a grid search for a polynomial kernel. I have used the normal grid search on polynomial kernels which is wrong. I only varied $c$ and $\gamma$... but I need to vary $a$ and $b$... Is there any fast method to do grid search for polynomial kernels?

Polynomial kernel: $K(x_i, x_j) = (\gamma x_i x_j + a)^b,\quad \gamma > 0$

Any suggestions or any corrections as well are welcome.

  • 1
    $\begingroup$ What is c? There's no c in your equation. $\endgroup$
    – onestop
    Mar 16 '12 at 17:58
  • $\begingroup$ C is the amount of misclassification.... it is for all kernels... $\endgroup$
    – lakesh
    Mar 16 '12 at 18:00

I woud recommend a grid search over integer values of b, and for each value of b perform a search for C, gamma and a using the Nelder-Mead simplex algorithm, which is a standard numerical optimiser that does not rely on gradient information and works fairly well (it is the fminsearch function in MATLAB).

  • $\begingroup$ I have one suggestion, not sure whether correct or wrong? Why don't u find the best value for c and gamma first, then with that value, try different values for b? $\endgroup$
    – lakesh
    Mar 16 '12 at 18:02
  • $\begingroup$ changing b changes the kernel induced feature space, you you can expect it to change the optimal value of the other hyper-parameters as well as they are often correlated. $\endgroup$ Mar 16 '12 at 18:06
  • $\begingroup$ if you show an example fo fminsearch using MATLAB, it would be really useful... thanks... $\endgroup$
    – lakesh
    Mar 16 '12 at 18:15
  • 1
    $\begingroup$ Without being familiar with the SVM package you are using I can't give any better information than the fminsearch help message. I use my own matlab software, which has model selection built in, but unfortunately has no manual, see theoval.cmp.uea.ac.uk/projects/gkm $\endgroup$ Mar 16 '12 at 18:25
  • $\begingroup$ i am using libsvm... $\endgroup$
    – lakesh
    Mar 16 '12 at 18:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.