6
$\begingroup$

Scikit learn support vector machine algorithm have a couple of coefficients which meaning I can not understand.

gamma : float, optional (default=0.0)

Kernel coefficient for ‘rbf’, ‘poly’ and ‘sigmoid’. If gamma is 0.0 then 1/n_features will be used instead.

and

coef0 : float, optional (default=0.0)

Independent term in kernel function. It is only significant in ‘poly’ and ‘sigmoid’.

I tried to google for gamma in support vector machines, but have not found anything relevant. Can anyone explain me what is the meaning of these parameters and reasonable value for them.

$\endgroup$
  • $\begingroup$ All told by Marc Claesen is true, but the question remains about poly type of kernels. In Math formula it seems that 'poly' is independent of 'gamma' while in practice results are different changing 'gamma'! $\endgroup$ – shining2014 Dec 24 '18 at 11:45
9
$\begingroup$

The RBF kernel function is as follows, for two vectors $\mathbf{u}$ and $\mathbf{v}$: $$ \kappa(\mathbf{u},\mathbf{v}) = \exp(-\gamma \|\mathbf{u}-\mathbf{v}\|^2). $$ The hyperparameter $\gamma$ is used to configure the sensitivity to differences in feature vectors, which in turn depends on various things such as input space dimensionality and feature normalization.

If you set $\gamma$ too large, you will end up overfitting. In the limit case $\gamma\rightarrow\infty$, the kernel matrix becomes the unit matrix which leads to a perfect fit of the training data, though an entirely useless model.

The optimal value of $\gamma$ depends entirely on your data, any rules of thumb should be taken with a pound of salt. That said, you can use specialized libraries to optimize hyperparameters for you (e.g. Optunity (*)), in the case of SVM with RBF kernel that is $\gamma$ and $C$. You can find an example of optimizing these parameters automatically with Optunity and scikit-learn here.

(*) disclaimer: I'm the lead developer of Optunity.

$\endgroup$
  • 1
    $\begingroup$ But in case of poly what does it mean, the polynomial kernel is just dependent upon the degree. Am I wrong? $\endgroup$ – hashcode55 Jul 14 '16 at 8:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.