2
$\begingroup$

The polynomial kernel is sometimes defined as just: $$ K(x,y):=(\left<x,y\right>+c)^d $$ with two parameters: the degree $d$ and constant coefficient $c$.

But others (e.g., libsvm, and sklearn which uses libsvm) include a scaling parameter $\gamma$: $$ K^\prime(x,y):=(\gamma \left<x,y\right>+r)^d $$

Now it's fairly easy to see that this is redundant. Choose $r=\gamma c$, and we get $$ K^\prime(x,y):=(\gamma \left<x,y\right>+\gamma c)^d=\gamma^d(\left<x,y\right>+c)^d=\gamma^d K(x,y) $$

But a constant scaling factor on the kernel does not matter for SVMs, does it?

I can imagine the following arguments: (1) consistency with RBF, where $\gamma$ is essential to scale the Gaussians (but I doubt you can choose the same value for both); (2) to avoid certain numerical range problems (c.f., libsvm guide mentions that the polynomial kernel can be problematic, but so can RBF) (3) other algorithms that do not learn weights - but which other algorithms work well with kernels and do not have weights to optimize?

Is there any theoretical reason why we should include $\gamma$ when teaching kernels?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.