3
$\begingroup$

Say I have training data $S_n$ and each point is of the form $x = \langle x_1 , x_2 \rangle$ in the original space (i.e. $x^{(i)} \in \mathbb{R}^2$).

I was considering the following kernel:

$$ K(x,x') = \frac{x^Tx}{|| x ||_{ \mathbb{R}^2 } || x' ||_{ \mathbb{R}^2 } } $$

Which has the following feature vector:

$$ \phi(x) = \frac{x}{|| x ||} $$

I was training to find an intuitive way to draw the decision boundary in the original space compared to the new data space.

This is what it should look like for a simple example:

enter image description here

The trouble that I have is visualizing and understanding rigorously how the decision boundary in the feature space becomes the lines in the original space.

I notice the pattern that is done and where ever the decision boundary intersects the unit circle, is where the line is drawn in the original space.

Somebody have a rigorous justification of this? I can see it in the picture but find it hard to generalize to higher dimensions.

$\endgroup$
1
$\begingroup$

A simple alternative method is to simulate your model on a 2-D grid and plot the contour $f(\mathbf{x})=0$. This is the decision boundary.

Of course, this is far less efficient than a closed form solution (in this case), but you can do this for any kernel function.

$\endgroup$
0
+50
$\begingroup$
  1. Note that the image of $\phi$ is the unit circle $C$ centered at zero. So for any $y \in C$ $\phi^{-1}(y) = \{ay|a \in R, a \neq 0\}$, while for any $y \notin C$ $\phi^{-1}(y) = \emptyset$.

  2. Let's rephrase your question. If we move 4 blue points around while keeping 2 red points fixed, when it's possible to separate the images of these 6 points under $\phi$ with a straight line? The answer is: it's always possible, as long as the 4 blue points do not fall on the line $y=x, x>0$, passing through the red points.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.