2
$\begingroup$

I am new to machine learning. Could anyone tell me the difference between linear kernel vs. polynomial kernel of degree 1 wrt SVM (if there is any difference)?

The reason I asked, I am getting different accuracy for both on the spam dataset from UCI.

Improve this question
$\endgroup$
0
1
$\begingroup$

Depending on your SVM implementation, there may be a difference. Compared to the linear kernel, the polynomial kernel has an additional parameter $c$ (and $d$ of course):

$K(x,y) = (x^\mathsf{T} y + c)^{d}$

However, if $c=0$ (and $d=1$), then:

(linear) $K(x,y) = x^\mathsf{T} y \equiv K(x,y) = (x^\mathsf{T} y + c)^{d}$ (polynomial)

Nonetheless, some SVM implementations may opt-out of calculating $c$ and assume homogeneity in order to reduce the number of hyper-parameters. So, it depends on how your SVM is implemented.

For example sklearn's polynomial kernel assumes by default $c=1$ if you do not specify the parameter, which would make it not exactly the same as a linear kernel.

Improve this answer
$\endgroup$
0
$\begingroup$

There can be a difference, as the Polynomial kernel has an additional parameter. If you're including that as a hyper-parameter, it could explain the difference.

Otherwise, if it is fixed to 0, then they should be the same. How big a difference in accuracy are you talking? If it's fairly minor it could be due to numerical stability issues.

Improve this answer
$\endgroup$
2
  • $\begingroup$ What is that additional parameter ? $\endgroup$ – meh May 30 '18 at 12:53
  • $\begingroup$ To me, a linear kernel would mean a separating hyper plane and that doesn't necessarily go through the origin. I wonder if this isn't something with the software. I know that Python likes to use regularization by default in some of it's ML software (e.g. logistic regression). That is a possible source of the difference. $\endgroup$ – meh May 30 '18 at 20:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.