# Support vector classifier / soft margin classifier

Is support vector machine with linear kernel the same as a soft margin classifier?

The idea of soft margin classifier means that one allows a number of misclassifications on the wrong part of the margin. This has nothing to do with the used kernel. Using a linear kernel or other type of kernel only affects in which kind of space the separating margin is searched. All SVMs uses soft margin, but one can push the soft margin behavior towards a hard margin behavior but imposing a very large penalty on errors.

[Later edit:]

Linear SVMs either for regression or for classification have a special characteristic. The numerical computations are simpler because one can compress the original result of the algorithm into a much simpler formula (due to it's linearity). This is why often implementations for SVMs and SVCs have sometimes separate implementations for the linear kernel.

• Thank you very much for your answer. So I have the task to train an Support Vector Classifier on my training data , an svm with RBF and svm with polynomial kernel. If I use the 'svmLinear' method in caret for the SVC am I correct or should I look for another procedure? Cause from the question I understand that although SVMs are based on soft margin, I have to follow a different procedure for the SVC...
– Kkk
Apr 5, 2021 at 18:01
• SVM is a generic name for the algorithm. When it does classification sometimes it is named SVC and when it does regression sometimes it is named SRV. You can't build a SVC/SRV with RBF or poly kernels with the svmLinear. That implementation is specific for linear kernel. Apr 5, 2021 at 18:28
• rstudio-pubs-static.s3.amazonaws.com/…
– Kkk
Apr 5, 2021 at 18:29
• Just found this, the question mentions support vector classifier and then solves it as svm with linear kernel and compares it with the svm radial and polynomial
– Kkk
Apr 5, 2021 at 18:30
• As I said, svm is sometimes named in implementations as svc (in sklearn for example) when it is used for classiffication and sometimes not. The same for regression. Apr 5, 2021 at 18:32

No. The two are independent of each other.

The kernel $$K(x, z)$$ is the generalisation of the scalar (dot) product, $$x \cdot z$$. We often choose $$K(x, z)$$ to be a non-linear function (polynomial, RBF, etc.). If not, i.e. if $$K(x, z) = x \cdot z$$, we say that the kernel is linear.

Independently of the kernel, a classifier can use a hard or a soft margin. A hard-margin classifier requires the classes to be linearly separable in the kernel-induced feature space. For the linear kernel this is the same as simply saying that the classes are linearly separable, but a non-linear kernel can also transform non-separable data into separable ones.

A soft-margin classifier can tolerate misclassification of some observations, making it suitable for training even if the data are not separable.

So, the choice of the kernel and the choice between the hard or soft margin are two distinct choices you need to make. In practice we almost always use soft margin, but control for its "softness" using the parameter $$C$$. As $$C \rightarrow \infty$$, the margin approaches the hard margin.