# Transform data to a higher dimensional space

The classes are as follows:

$C_1=\{3, 3.5, 4, 4.5, 5, 5.5, 7\} \cup \{15, 16, 17\}$

$C_2=\{0,0.5,1,2\}\cup\{8,9,10,11,12,13\}\cup\{20,25,30\}$

And we wish to classify numbers from the interval $[0,40]$. I think since they are not linearly separable (the two classes in a one dimensional input space) so they must be transformed to a higher dimensional space where they are linearly separable. Will that idea go anywhere? If yes, how should I do it?

I want to use SVM as a classification method to classify a given number on that interval but I don't seem to understand what has to be done in this case even though I am well aware of the theoretical idea behind SVM.

A general comment: if you want to make two classes of points linearly separable by mapping them into another space, that does not mean that your new space should necessarily have more dimensions than the original space. It may have the same or even smaller number of dimensions.

In your particular example, it's easy to notice that Class 1 consists of two clouds of points: one is centered around $p_1 = 4.6$, and another one is centered around $p_2 = 16.0$. This suggests considering, say, the following mapping:

$x_{\mathrm{new}} = |x - p_1|\times|x - p_2|$,

which will make your (training) data linearly separable.

The problem with such a "manual" approach is that when you have complex high-dimensional data, it's very difficult to figure out its structure and to come up with simple linearization mappings. The generalization ability of a classifier trained on transformed data is another important aspect to consider.

• Could you explain to me how did you come up with that mapping? Why the multiplication? I mean, is it something that you just guessed or does it have a mathematical reason? Nov 19 '12 at 3:20
• I guessed it based on the structure of the given data.
– Leo
Nov 19 '12 at 22:27
• May I know which software you used to draw those graphs? Dec 1 '12 at 22:30
• Sure, I used R.
– Leo
Dec 3 '12 at 20:04

You've got the right idea, but I've used a fair number of SVM packages (Weka, LibSVM, PyML, etc) and none of these require you to "pre-transform" your data into a higher dimensional space. In fact, the whole idea behind the "kernel trick" is that you don't actually have to compute the mapping explicitly. You just need to load in your data, choose your kernel and hyper-parameters, and you're off!

• Thank you for your answer. You're probably right but I am supposed to do it by hand first and then write a matlab program for it! So I think I still need to understand how to transfrom the data to a two or more dimensional space. Nov 17 '12 at 21:36
• Perhaps this example from Michael I. Jordon's lecture notes will help? cs.berkeley.edu/~jordan/courses/281B-spring04/lectures/lec3.pdf Nov 17 '12 at 21:46