How do you transform a decision boundary in the angle kernel to the original space?

Question

Say I have training data $S_n$ and each point is of the form $x = \langle x_1 , x_2 \rangle$ in the original space (i.e. $x^{(i)} \in \mathbb{R}^2$).

I was considering the following kernel:

$$ K(x,x') = \frac{x^Tx}{|| x ||_{ \mathbb{R}^2 } || x' ||_{ \mathbb{R}^2 } } $$

Which has the following feature vector:

$$ \phi(x) = \frac{x}{|| x ||} $$

I was training to find an intuitive way to draw the decision boundary in the original space compared to the new data space.

This is what it should look like for a simple example:

The trouble that I have is visualizing and understanding rigorously how the decision boundary in the feature space becomes the lines in the original space.

I notice the pattern that is done and where ever the decision boundary intersects the unit circle, is where the line is drawn in the original space.

Somebody have a rigorous justification of this? I can see it in the picture but find it hard to generalize to higher dimensions.

Answer 1

A simple alternative method is to simulate your model on a 2-D grid and plot the contour $f(\mathbf{x})=0$. This is the decision boundary.

Of course, this is far less efficient than a closed form solution (in this case), but you can do this for any kernel function.

Answer 2

1. Note that the image of $\phi$ is the unit circle $C$ centered at zero. So for any $y \in C$ $\phi^{-1}(y) = \{ay|a \in R, a \neq 0\}$, while for any $y \notin C$ $\phi^{-1}(y) = \emptyset$.

2. Let's rephrase your question. If we move 4 blue points around while keeping 2 red points fixed, when it's possible to separate the images of these 6 points under $\phi$ with a straight line? The answer is: it's always possible, as long as the 4 blue points do not fall on the line $y=x, x>0$, passing through the red points.