I would like to use a SVM classifier with a sigmoid and polynomial kernel.

The sigmoid kernel has the following form: $K(u,v) = \tanh(\gamma * u'v + \text{coef}_{0})$

The polynomial kernel has the following form: $K(u,v) = (\gamma * u'v + \text{coef}_{0})^d$

Which parameter ranges are reasonable to try for $\gamma$ and $\text{coef}_{0}$?

  • $\begingroup$ All parameter values (and ranges) are "wrong". Use caret::train or a similar package to get values based on some cross-validation metrics. Choosing hyperparameters is voodoo. $\endgroup$
    – usεr11852
    May 6 '16 at 19:10
  • $\begingroup$ @usεr11852 I would like to choose the hyperparameters by cross-validation but for this I have to know the ranges to span the grid over. $\endgroup$
    – BlackHawk
    May 6 '16 at 19:52
  • $\begingroup$ This will depend on the range of the arguments $u$ an $v$. This is why in most cases you normalise your features before feeding them in a algorithm. In general for the case of $\tanh$ you just want to make sure that you have good coverage over [0,1]. For the polynomial kernel you practically want to avoid under- or over-flow. See Bergstra and Bengio, 2012 Random search for hyper-parameter optimization for more. $\endgroup$
    – usεr11852
    May 6 '16 at 21:10
  • $\begingroup$ @usεr11852 Yes, I have normalized my dataset. In the given paper they did not propose ranges and for applying random search I have to now some upper and lower bounds (= ranges) for the parameters.. $\endgroup$
    – BlackHawk
    May 6 '16 at 21:14
  • $\begingroup$ Yes, pick them such that you do have reasonable coverage. You don't want to use $\text{coef}_0$ extremely high for example, this will just limit your $K(\bullet)$ estimates to very high values only. You should start with a coarse grid and then move to a finer grid. $\endgroup$
    – usεr11852
    May 6 '16 at 21:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.