# The difference of kernels in SVM?

Can someone please tell me the difference between the kernels in SVM:

1. Linear
2. Polynomial
3. Gaussian (RBF)
4. Sigmoid

Because as we know that kernel is used to mapped our input space into high dimensionality feature space. And in that feature space, we find the linearly separable boundary..

When are they are used (under what condition) and why?

• "as we know that kernel is used to map our input space into high dimensionality feature space" not clear. More on kernel functions. The aimed space is actually one with enough dimensions to transform (bend) the input space so that the classifier can now find the boundaries it needs. The kernel is the function performing such transform.
– mins
Jan 31, 2021 at 16:50
• This answers your questions exactly. However there are other SVMs, chi-squared, logit (as apposed to logistic regression), etc., community.ibm.com/community/user/ai-datascience/blogs/… You should Google for a book on support vector machines, each university has their own pdf. timofey.pro/static/pdfdocs/… I hope that helps. Mar 26 at 19:26

The linear kernel is what you would expect, a linear model. I believe that the polynomial kernel is similar, but the boundary is of some defined but arbitrary order

(e.g. order 3: $$a= b_1 + b_2 \cdot X + b_3 \cdot X^2 + b_4 \cdot X^3$$).

RBF uses normal curves around the data points, and sums these so that the decision boundary can be defined by a type of topology condition such as curves where the sum is above a value of 0.5. (see this picture ) I am not certain what the sigmoid kernel is, unless it is similar to the logistic regression model where a logistic function is used to define curves according to where the logistic value is greater than some value (modeling probability), such as 0.5 like the normal case.

• So, we can get the linierly separable hyperplane (boundary),if we use the linear kernel?? and if we use polynomial or RBF kernel, the (for polynomial)hyperlane could be a circle of grouped classes (for RBF) and curve ?? is that right??scikit-learn.org/stable/modules/svm.html Mar 21, 2014 at 4:33
• Each of the kernals work for high dimension versions of their respective boundaries. Does that answer your question? You are not limited to three dimensions for any kernel that I am aware of. Mar 21, 2014 at 4:37
• I just wanna make it clear. So the boundary by using linear kernel is a linear? For RBF is like a circle of grouped class?? and for polynomial, it can be curve based on the degree of the polynomial?? Mar 21, 2014 at 4:48
• I would not say RBF is a circle of grouped class. My understanding is that it applies a function based upon a normal distribution at each data point, and sums these functions. Then a boundary is formed by the curve representing a certain value on that function. If someone who has contributed to an SVM library could chime in, that might help. I think that your understanding of the other two kernels is correct. Mar 21, 2014 at 5:01
• U said that Linier Kernel is what I expected (to get linierly separable class) by using Kernel. and by using SVM classifier, we called it LINIER SVM. But how if we can get the linierly separable data without any kernel in SVM. What we call it?? Still Linier SVM or Non Linier SVM?? Mar 22, 2014 at 9:24

Linear Kernel: $K(X, Y) = X^T Y$

Polynomial kernel: $K(X, Y) = (γ\cdot X^T Y + r)^d , γ > 0$

Radial basis function (RBF) Kernel: $K(X, Y) = \exp(\|X-Y\|^2/2σ^2)$ which in simple form can be written as $\exp(-γ \cdot \|X - Y\|^2), γ > 0$

Sigmoid Kernel: $K(X, Y) = \tanh(γ\cdot X^TY + r)$ which is similar to the sigmoid function in logistic regression.

Here $r$, $d$, and $γ$ are kernel parameters.

• While the information in your answer is correct, I don't think it answers the question here raised, which is more towards what's the practical difference between them, i.e. when to use one or the other. Oct 29, 2016 at 19:25
• Amazingly these simple definitions are hard to come by. They should be the first thing presented when talking about differences of kernels, yet there is a wide-spread failure to state them. Sep 9, 2017 at 16:22
• Is there any official source for these? (I tested them and they seem correct, but I'd like to be able to cite them.) May 15, 2018 at 18:16
• can use for ciitation Jun 24 at 15:33
• @Firebug: here in "Types of SVM kernels" section can see "when to use one or the other" Jun 24 at 15:43

This question can be answered from theoretical and practical point of view. From theoretical according to No-Free Lunch theorem states that there are no guarantees for one kernel to work better than the other. That is a-priori you never know nor you can find out which kernel will work better.

From practical point of view consult the following page:

How to select kernel for SVM?

While reflecting on what a kernel is "good for" or when it should be used, there are no hard and fast rules.

If you're classifier/regressor is performing well with a given kernel, it is appropriate, if not, consider changing to another.

Insight into how your kernel may perform, specifically if it is a classification model, might be gained by reviewing some visualisation examples, e.g. https://gist.github.com/WittmannF/60680723ed8dd0cb993051a7448f7805