8
$\begingroup$

enter image description here

Ref to above image, clearly a circle can separate the two classes(left image). Why then take so much pain to map it to a function to make it linearly separable (right image) ?

Can anyone please explain ? I really couldn't find anything on the web or youtube lectures on the why

$\endgroup$

5 Answers 5

5
$\begingroup$

Well, that is the whole idea behind support vector machines! svm are searching for a hyperplane that separates the classes (why the name), and that can of course be done most effectively it the points are linearly separable (that's not a deep point, it is a summary of the full idea). In the example you show, point lie on concentric annular rings, which cannot be separated by any plane, but by introducing a new variable RADIUS---distance from center---you get complete linear separation.

$\endgroup$
4
  • $\begingroup$ You mean to say that linear seperability of classes is better/easier than non-linear seperability? $\endgroup$
    – vinita
    Jul 31, 2017 at 15:07
  • 1
    $\begingroup$ Classes can be non-linearly separable in an infinite number of ways, so yes, linear separation is clearly easier to handle! And, that is the complete idea behind SVM, so that they work better for data which comply with the asumptions behind the method should hardly be a surprise ... $\endgroup$ Jul 31, 2017 at 15:09
  • $\begingroup$ @kjetilbhalvorsen I think the key idea behind SVM is kernel trick to save time for computation. But not "using polynomial basis expansion". $\endgroup$
    – Haitao Du
    Jul 31, 2017 at 15:11
  • 1
    $\begingroup$ @hdx1101 - The kernel trick makes a lot of stuff feasible, computationally, but that's a big boon to implementation, not the idea behind the method itself. $\endgroup$
    – jbowman
    Jul 31, 2017 at 19:45
2
$\begingroup$

Why is it desirable to have linear separability in SVM?

SVCs are inherently a linear technique. They find linear boundaries separating (as best possible) different classes. If there is no natural linear boundary for the problem, the choices are either to use a different technique, or to use SVCs with transformed features into a space where there indeed is a linear boundary.

Ref to above image, clearly a circle can separate the two classes(left image). Why then take so much pain to map it to a function to make it linearly separable (right image) ?

This is a classic example. The data classes are separated by a circle, but an SVC cannot find circles directly. However, if the data are transformed using a radial basis function, then in the resulting space, the classes are separated by a linear boundary.

$\endgroup$
0
$\begingroup$

Not directly answer your question but,

It is important to keep in mind the difference between basis expansion and Kernel method / SVM.

  • We can "expand data" using basis expansion in different ways. For example, polynomial expansion, splines, Fourier series, etc. These basis expansion have little to do with SVM, kernel trick.

  • SVM with polynomial kernel provides use a "computational effect" ways to do polynomial basis expansion. Search Kernel trick for details.

$\endgroup$
0
$\begingroup$

SVCs were originally defined to be linear classifiers which tried to find maximum margin hyperplanes. The optimization problem is derived starting with an equation of a plane and calculating the distance of points from it, and finding out its parameter matrix w. See here for details. In a 2 dimensional case(left figure) a 1D hyperplane i.e. a straight line clearly can't separate the two classes, but with careful projection of data into 3rd dimension, support vectors are easily separated by a 2D hyperplane, as for these points x^2 + y^2 is near constant.

$\endgroup$
-2
$\begingroup$

You are correct. When the field says "linearly separable", they mean that the data should be "differentiable": that there exists some filtering function that you can overlay onto the dataset to create two or more distinct groupings (with some small error tolerance).

That's all. But you should point out to the academics to clean up thier language.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.