0
$\begingroup$

I was reading that the Gaussian/RBF kernel maps its input onto the surface of normalized hypersphere.

Our RBF kernel given by:

$k(x,z) = exp(\frac{- ||x-z||^2}{2\sigma^2})$

Can anyone explain why the RBF kernel maps the input space onto the surface of a unit hypersphere?

$\endgroup$

2 Answers 2

2
$\begingroup$

RBF kernel's explicit feature map, which isn't unique, (as given in Slide 11 of this link) is (it gives the mapping only for for 1D case): $$\phi(x)=e^{-\gamma x^2}\left[1,\sqrt{\frac{2\gamma}{1!}}x,\sqrt{\frac{(2\gamma)^2}{2!}}x^2,...\right]$$

If we calculate the Euclidean norm of this vector, we'd have: $$\Vert\phi(x)\Vert^2=e^{-2\gamma x^2}\sum_{i=0}^\infty \frac{(2\gamma)^i}{i!}x^{2i}=e^{-2\gamma x^2}\sum_{i=0}^\infty\frac{(2\gamma x^2)^{i}}{i!}=e^{-2\gamma x^2}e^{2\gamma x^2}=1$$

So, the norm of the vectors are $1$ ($\Vert\phi(x)\Vert^2=1\rightarrow \Vert\phi(x)\Vert=1)$, which means the mapping is onto surface of unit hypersphere in infinite dimensions. I'd gladly try to prove this for larger dimensions if I can find an explicit expression.

$\endgroup$
0
1
$\begingroup$

for x of any dimension:

we know that $\phi (x_1) \cdot \phi (x_2) = k(x_1, x_2)$, so, it's clear that $||\phi(x)||^2 = k(x, x)$. we can now try to find the norm of generic $\phi(x)$ and, if we find it to be constant, this means that RBF kernel maps x in a hypersphere.

$$||\phi(x)||^2 = k(x, x) = e^{-\gamma ||x-x||^2} = e^0 = 1$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.