3
$\begingroup$

The classes are as follows:

$C_1=\{3, 3.5, 4, 4.5, 5, 5.5, 7\} \cup \{15, 16, 17\}$

$C_2=\{0,0.5,1,2\}\cup\{8,9,10,11,12,13\}\cup\{20,25,30\}$

And we wish to classify numbers from the interval $[0,40]$. I think since they are not linearly separable (the two classes in a one dimensional input space) so they must be transformed to a higher dimensional space where they are linearly separable. Will that idea go anywhere? If yes, how should I do it?

I want to use SVM as a classification method to classify a given number on that interval but I don't seem to understand what has to be done in this case even though I am well aware of the theoretical idea behind SVM.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
11
$\begingroup$

A general comment: if you want to make two classes of points linearly separable by mapping them into another space, that does not mean that your new space should necessarily have more dimensions than the original space. It may have the same or even smaller number of dimensions.

In your particular example, it's easy to notice that Class 1 consists of two clouds of points: one is centered around $p_1 = 4.6$, and another one is centered around $p_2 = 16.0$. This suggests considering, say, the following mapping:

$x_{\mathrm{new}} = |x - p_1|\times|x - p_2|$,

which will make your (training) data linearly separable.

enter image description here

The problem with such a "manual" approach is that when you have complex high-dimensional data, it's very difficult to figure out its structure and to come up with simple linearization mappings. The generalization ability of a classifier trained on transformed data is another important aspect to consider.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Could you explain to me how did you come up with that mapping? Why the multiplication? I mean, is it something that you just guessed or does it have a mathematical reason? $\endgroup$
    – Gigili
    Nov 19, 2012 at 3:20
  • $\begingroup$ I guessed it based on the structure of the given data. $\endgroup$
    – Leo
    Nov 19, 2012 at 22:27
  • $\begingroup$ May I know which software you used to draw those graphs? $\endgroup$
    – Gigili
    Dec 1, 2012 at 22:30
  • $\begingroup$ Sure, I used R. $\endgroup$
    – Leo
    Dec 3, 2012 at 20:04
2
$\begingroup$

You've got the right idea, but I've used a fair number of SVM packages (Weka, LibSVM, PyML, etc) and none of these require you to "pre-transform" your data into a higher dimensional space. In fact, the whole idea behind the "kernel trick" is that you don't actually have to compute the mapping explicitly. You just need to load in your data, choose your kernel and hyper-parameters, and you're off!

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for your answer. You're probably right but I am supposed to do it by hand first and then write a matlab program for it! So I think I still need to understand how to transfrom the data to a two or more dimensional space. $\endgroup$
    – Gigili
    Nov 17, 2012 at 21:36
  • $\begingroup$ Perhaps this example from Michael I. Jordon's lecture notes will help? cs.berkeley.edu/~jordan/courses/281B-spring04/lectures/lec3.pdf $\endgroup$
    – Matt Krause
    Nov 17, 2012 at 21:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.