The polynomial kernel is sometimes defined as just: $$ K(x,y):=(\left<x,y\right>+c)^d $$ with two parameters: the degree $d$ and constant coefficient $c$.

But others (e.g., libsvm, and sklearn which uses libsvm) include a scaling parameter $\gamma$: $$ K^\prime(x,y):=(\gamma \left<x,y\right>+r)^d $$

Now it's fairly easy to see that this is redundant. Choose $r=\gamma c$, and we get $$ K^\prime(x,y):=(\gamma \left<x,y\right>+\gamma c)^d=\gamma^d(\left<x,y\right>+c)^d=\gamma^d K(x,y) $$

But a constant scaling factor on the kernel does not matter for SVMs, does it?

I can imagine the following arguments: (1) consistency with RBF, where $\gamma$ is essential to scale the Gaussians (but I doubt you can choose the same value for both); (2) to avoid certain numerical range problems (c.f., libsvm guide mentions that the polynomial kernel can be problematic, but so can RBF) (3) other algorithms that do not learn weights - but which other algorithms work well with kernels and do not have weights to optimize?

Is there any theoretical reason why we should include $\gamma$ when teaching kernels?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.