0
$\begingroup$

Im trying to learn some hyper-parameters for SVM classifier,

I want to know if there is any correlation between the kernel parameters and the regularization parameter - C,. because if not i can then try optimizing the C parameter and only when one has being optimized start with the kernel parameter, which will save me alot of runtime.

$\endgroup$
1
  • $\begingroup$ what is "kernel parameter"? $\endgroup$
    – Haitao Du
    Dec 21 '17 at 15:09
1
$\begingroup$

In principle, no. One cannot optimize one parameter and then the other.

There is (at least) one paper that proposes a method to optimize first the C (using a Linear SVM) and then the gamma.

http://www.mitpressjournals.org/doi/abs/10.1162/089976603321891855#.WE3VlpJrWLA

but I tried this and it did not work well on many datasets. Two problems (a) the selection it makes is not that great and (b) it takes a surprising long time - because the linear SVM is not that fast (I did not use the LibLinear implementation - I used libSVM with the linear kernel).

$\endgroup$
1
$\begingroup$

They are indeed correlated, the complexity of the model can be increased by making the kernel more sensitive, which may mean more regularisation is required. Alternatively if you use a "bland" kernel, there is usually less need for regularisation. In short, the optimal value of the regularisation parameter depends on the values of the kernel parameters, and vice versa, so you need to tune both (I tend to use the Nelder-Mead simplex method, which is generally cheaper than grid-search).

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.