The polynomial kernel is sometimes defined as just: $$ K(x,y):=(\left<x,y\right>+c)^d $$ with two parameters: the degree $d$ and constant coefficient $c$.

But others (e.g., libsvm, and sklearn which uses libsvm) include a scaling parameter $\gamma$: $$ K^\prime(x,y):=(\gamma \left<x,y\right>+r)^d $$

Now it's fairly easy to see that this is redundant. Choose $r=\gamma c$, and we get $$ K^\prime(x,y):=(\gamma \left<x,y\right>+\gamma c)^d=\gamma^d(\left<x,y\right>+c)^d=\gamma^d K(x,y) $$

But a constant scaling factor on the kernel does not matter for SVMs, does it?

I can imagine the following arguments: (1) consistency with RBF, where $\gamma$ is essential to scale the Gaussians (but I doubt you can choose the same value for both); (2) to avoid certain numerical range problems (c.f., libsvm guide mentions that the polynomial kernel can be problematic, but so can RBF) (3) other algorithms that do not learn weights - but which other algorithms work well with kernels and do not have weights to optimize?

Is there any theoretical reason why we should include $\gamma$ when teaching kernels?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.