3
$\begingroup$

By the reproducing kernel Hilbert space (RKHS) property, given a P.S.D. kernel function $\kappa:X\times X \rightarrow \mathbb R$, there exists a Hilbert space $H$ and a map $\phi:X\rightarrow H$ such that $$ \kappa(x,y) = \langle \phi(x), \phi(y) \rangle_H. $$

Define $$ \kappa(x,y) = \exp\left(\frac{-\|x-y\|^2}{2\sigma^2}\right) $$ and denote by $H$ the corresponding RKHS.

The definition of the kernel implies that $H$ is defined over field $\mathbb R$. By the properties of the inner product, it should be that for any $\alpha \in \mathbb R$, $$ \langle \alpha\phi(x), \phi(y) \rangle_H = \alpha\langle \phi(x), \phi(y) \rangle_H. $$ This means that we can choose vectors $\phi(x),\phi(y) \in H$ to achieve any real value as a result of $\kappa(x,y)$. However, the image of $\kappa$ is $(0,1]$, so I guess that something must be off in my reasoning.

What is it?

$\endgroup$

1 Answer 1

2
$\begingroup$

It is true that $\kappa(x, y) = \langle \phi(x), \phi(y) \rangle_H$. So for any $x$, $y$, with this choice of $\kappa$, $\kappa(x, y) \in (0, 1]$.

It is also true that, say, $-\phi(x) \in H$, and $\langle - \phi(x), \phi(y) \rangle_H = - k(x, y) \in [-1, 0)$.

It is not true that there is necessarily some $x'$ such that $-\phi(x) = \phi(x')$. The feature mapping $\phi$ doesn't cover all of $H$.

$\endgroup$
1
  • 1
    $\begingroup$ Wonderfully simple, now that I think of it. An immediate insight, which I'd never stopped to think about before now, is that this kernel is defined only for the points on the unit sphere in $H$. $\endgroup$
    – cangrejo
    Commented Nov 1, 2019 at 9:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.