# The inner product properties seem to clash with the RKHS property for RBF kernels. What is off?

By the reproducing kernel Hilbert space (RKHS) property, given a P.S.D. kernel function $$\kappa:X\times X \rightarrow \mathbb R$$, there exists a Hilbert space $$H$$ and a map $$\phi:X\rightarrow H$$ such that $$\kappa(x,y) = \langle \phi(x), \phi(y) \rangle_H.$$

Define $$\kappa(x,y) = \exp\left(\frac{-\|x-y\|^2}{2\sigma^2}\right)$$ and denote by $$H$$ the corresponding RKHS.

The definition of the kernel implies that $$H$$ is defined over field $$\mathbb R$$. By the properties of the inner product, it should be that for any $$\alpha \in \mathbb R$$, $$\langle \alpha\phi(x), \phi(y) \rangle_H = \alpha\langle \phi(x), \phi(y) \rangle_H.$$ This means that we can choose vectors $$\phi(x),\phi(y) \in H$$ to achieve any real value as a result of $$\kappa(x,y)$$. However, the image of $$\kappa$$ is $$(0,1]$$, so I guess that something must be off in my reasoning.

What is it?

It is true that $$\kappa(x, y) = \langle \phi(x), \phi(y) \rangle_H$$. So for any $$x$$, $$y$$, with this choice of $$\kappa$$, $$\kappa(x, y) \in (0, 1]$$.
It is also true that, say, $$-\phi(x) \in H$$, and $$\langle - \phi(x), \phi(y) \rangle_H = - k(x, y) \in [-1, 0)$$.
It is not true that there is necessarily some $$x'$$ such that $$-\phi(x) = \phi(x')$$. The feature mapping $$\phi$$ doesn't cover all of $$H$$.
• Wonderfully simple, now that I think of it. An immediate insight, which I'd never stopped to think about before now, is that this kernel is defined only for the points on the unit sphere in $H$. Nov 1, 2019 at 9:55