3
$\begingroup$

The classes are as follows:

$C_1=\{3, 3.5, 4, 4.5, 5, 5.5, 7\} \cup \{15, 16, 17\}$

$C_2=\{0,0.5,1,2\}\cup\{8,9,10,11,12,13\}\cup\{20,25,30\}$

And we wish to classify numbers from the interval $[0,40]$. I think since they are not linearly separable (the two classes in a one dimensional input space) so they must be transformed to a higher dimensional space where they are linearly separable. Will that idea go anywhere? If yes, how should I do it?

I want to use SVM as a classification method to classify a given number on that interval but I don't seem to understand what has to be done in this case even though I am well aware of the theoretical idea behind SVM.

$\endgroup$
11
$\begingroup$

A general comment: if you want to make two classes of points linearly separable by mapping them into another space, that does not mean that your new space should necessarily have more dimensions than the original space. It may have the same or even smaller number of dimensions.

In your particular example, it's easy to notice that Class 1 consists of two clouds of points: one is centered around $p_1 = 4.6$, and another one is centered around $p_2 = 16.0$. This suggests considering, say, the following mapping:

$x_{\mathrm{new}} = |x - p_1|\times|x - p_2|$,

which will make your (training) data linearly separable.

enter image description here

The problem with such a "manual" approach is that when you have complex high-dimensional data, it's very difficult to figure out its structure and to come up with simple linearization mappings. The generalization ability of a classifier trained on transformed data is another important aspect to consider.

$\endgroup$
4
  • $\begingroup$ Could you explain to me how did you come up with that mapping? Why the multiplication? I mean, is it something that you just guessed or does it have a mathematical reason? $\endgroup$
    – Gigili
    Nov 19 '12 at 3:20
  • $\begingroup$ I guessed it based on the structure of the given data. $\endgroup$
    – Leo
    Nov 19 '12 at 22:27
  • $\begingroup$ May I know which software you used to draw those graphs? $\endgroup$
    – Gigili
    Dec 1 '12 at 22:30
  • $\begingroup$ Sure, I used R. $\endgroup$
    – Leo
    Dec 3 '12 at 20:04
2
$\begingroup$

You've got the right idea, but I've used a fair number of SVM packages (Weka, LibSVM, PyML, etc) and none of these require you to "pre-transform" your data into a higher dimensional space. In fact, the whole idea behind the "kernel trick" is that you don't actually have to compute the mapping explicitly. You just need to load in your data, choose your kernel and hyper-parameters, and you're off!

$\endgroup$
2
  • $\begingroup$ Thank you for your answer. You're probably right but I am supposed to do it by hand first and then write a matlab program for it! So I think I still need to understand how to transfrom the data to a two or more dimensional space. $\endgroup$
    – Gigili
    Nov 17 '12 at 21:36
  • $\begingroup$ Perhaps this example from Michael I. Jordon's lecture notes will help? cs.berkeley.edu/~jordan/courses/281B-spring04/lectures/lec3.pdf $\endgroup$ Nov 17 '12 at 21:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.