RBF kernel mapping

Question

I was reading that the Gaussian/RBF kernel maps its input onto the surface of normalized hypersphere.

Our RBF kernel given by:

$k(x,z) = exp(\frac{- ||x-z||^2}{2\sigma^2})$

Can anyone explain why the RBF kernel maps the input space onto the surface of a unit hypersphere?

Answer 1

RBF kernel's explicit feature map, which isn't unique, (as given in Slide 11 of this link) is (it gives the mapping only for for 1D case): $$\phi(x)=e^{-\gamma x^2}\left[1,\sqrt{\frac{2\gamma}{1!}}x,\sqrt{\frac{(2\gamma)^2}{2!}}x^2,...\right]$$

If we calculate the Euclidean norm of this vector, we'd have: $$\Vert\phi(x)\Vert^2=e^{-2\gamma x^2}\sum_{i=0}^\infty \frac{(2\gamma)^i}{i!}x^{2i}=e^{-2\gamma x^2}\sum_{i=0}^\infty\frac{(2\gamma x^2)^{i}}{i!}=e^{-2\gamma x^2}e^{2\gamma x^2}=1$$

So, the norm of the vectors are $1$ ($\Vert\phi(x)\Vert^2=1\rightarrow \Vert\phi(x)\Vert=1)$, which means the mapping is onto surface of unit hypersphere in infinite dimensions. I'd gladly try to prove this for larger dimensions if I can find an explicit expression.

Answer 2

for x of any dimension:

we know that $\phi (x_1) \cdot \phi (x_2) = k(x_1, x_2)$, so, it's clear that $||\phi(x)||^2 = k(x, x)$. we can now try to find the norm of generic $\phi(x)$ and, if we find it to be constant, this means that RBF kernel maps x in a hypersphere.

$$||\phi(x)||^2 = k(x, x) = e^{-\gamma ||x-x||^2} = e^0 = 1$$