34
$\begingroup$

Can someone please tell me the difference between the kernels in SVM:

  1. Linear
  2. Polynomial
  3. Gaussian (RBF)
  4. Sigmoid

Because as we know that kernel is used to mapped our input space into high dimensionality feature space. And in that feature space, we find the linearly separable boundary..

When are they are used (under what condition) and why?

Improve this question
$\endgroup$
2
  • $\begingroup$ "as we know that kernel is used to map our input space into high dimensionality feature space" not clear. More on kernel functions. The aimed space is actually one with enough dimensions to transform (bend) the input space so that the classifier can now find the boundaries it needs. The kernel is the function performing such transform. $\endgroup$
    – mins
    Commented Jan 31, 2021 at 16:50
  • $\begingroup$ This answers your questions exactly. However there are other SVMs, chi-squared, logit (as apposed to logistic regression), etc., community.ibm.com/community/user/ai-datascience/blogs/… You should Google for a book on support vector machines, each university has their own pdf. timofey.pro/static/pdfdocs/… I hope that helps. $\endgroup$
    – user334498
    Commented Mar 26, 2023 at 19:26

4 Answers 4

Reset to default
9
$\begingroup$

The linear kernel is what you would expect, a linear model. I believe that the polynomial kernel is similar, but the boundary is of some defined but arbitrary order

(e.g. order 3: $ a= b_1 + b_2 \cdot X + b_3 \cdot X^2 + b_4 \cdot X^3$).

RBF uses normal curves around the data points, and sums these so that the decision boundary can be defined by a type of topology condition such as curves where the sum is above a value of 0.5. (see this picture )

enter image description here

I am not certain what the sigmoid kernel is, unless it is similar to the logistic regression model where a logistic function is used to define curves according to where the logistic value is greater than some value (modeling probability), such as 0.5 like the normal case.

Improve this answer
$\endgroup$
6
  • $\begingroup$ So, we can get the linierly separable hyperplane (boundary),if we use the linear kernel?? and if we use polynomial or RBF kernel, the (for polynomial)hyperlane could be a circle of grouped classes (for RBF) and curve ?? is that right??scikit-learn.org/stable/modules/svm.html $\endgroup$
    – user3378327
    Commented Mar 21, 2014 at 4:33
  • $\begingroup$ Each of the kernals work for high dimension versions of their respective boundaries. Does that answer your question? You are not limited to three dimensions for any kernel that I am aware of. $\endgroup$
    – John Yetter
    Commented Mar 21, 2014 at 4:37
  • $\begingroup$ I just wanna make it clear. So the boundary by using linear kernel is a linear? For RBF is like a circle of grouped class?? and for polynomial, it can be curve based on the degree of the polynomial?? $\endgroup$
    – user3378327
    Commented Mar 21, 2014 at 4:48
  • $\begingroup$ I would not say RBF is a circle of grouped class. My understanding is that it applies a function based upon a normal distribution at each data point, and sums these functions. Then a boundary is formed by the curve representing a certain value on that function. If someone who has contributed to an SVM library could chime in, that might help. I think that your understanding of the other two kernels is correct. $\endgroup$
    – John Yetter
    Commented Mar 21, 2014 at 5:01
  • $\begingroup$ U said that Linier Kernel is what I expected (to get linierly separable class) by using Kernel. and by using SVM classifier, we called it LINIER SVM. But how if we can get the linierly separable data without any kernel in SVM. What we call it?? Still Linier SVM or Non Linier SVM?? $\endgroup$
    – user3378327
    Commented Mar 22, 2014 at 9:24
20
$\begingroup$

Relying on basic knowledge of reader about kernels.

Linear Kernel: $K(X, Y) = X^T Y$

Polynomial kernel: $K(X, Y) = (γ\cdot X^T Y + r)^d , γ > 0$

Radial basis function (RBF) Kernel: $K(X, Y) = \exp(\|X-Y\|^2/2σ^2)$ which in simple form can be written as $\exp(-γ \cdot \|X - Y\|^2), γ > 0$

Sigmoid Kernel: $K(X, Y) = \tanh(γ\cdot X^TY + r) $ which is similar to the sigmoid function in logistic regression.

Here $r$, $d$, and $γ$ are kernel parameters.

Improve this answer
$\endgroup$
6
  • 9
    $\begingroup$ While the information in your answer is correct, I don't think it answers the question here raised, which is more towards what's the practical difference between them, i.e. when to use one or the other. $\endgroup$
    – Firebug
    Commented Oct 29, 2016 at 19:25
  • 1
    $\begingroup$ Amazingly these simple definitions are hard to come by. They should be the first thing presented when talking about differences of kernels, yet there is a wide-spread failure to state them. $\endgroup$
    – cammil
    Commented Sep 9, 2017 at 16:22
  • $\begingroup$ Is there any official source for these? (I tested them and they seem correct, but I'd like to be able to cite them.) $\endgroup$
    – Christian Eriksson
    Commented May 15, 2018 at 18:16
  • $\begingroup$ can use for ciitation $\endgroup$
    – JeeyCi
    Commented Jun 24, 2023 at 15:33
  • $\begingroup$ @Firebug: here in "Types of SVM kernels" section can see "when to use one or the other" $\endgroup$
    – JeeyCi
    Commented Jun 24, 2023 at 15:43
13
$\begingroup$

This question can be answered from theoretical and practical point of view. From theoretical according to No-Free Lunch theorem states that there are no guarantees for one kernel to work better than the other. That is a-priori you never know nor you can find out which kernel will work better.

From practical point of view consult the following page:

How to select kernel for SVM?

Improve this answer
$\endgroup$
1
$\begingroup$

While reflecting on what a kernel is "good for" or when it should be used, there are no hard and fast rules.

If you're classifier/regressor is performing well with a given kernel, it is appropriate, if not, consider changing to another.

Insight into how your kernel may perform, specifically if it is a classification model, might be gained by reviewing some visualisation examples, e.g. https://gist.github.com/WittmannF/60680723ed8dd0cb993051a7448f7805

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.