1
$\begingroup$

A kernel, $k(x_1, x_2)$, has the interesting property that it may be represented as the dot product in a reproducing kernel hilbert space (RKHS), $\phi(x_0)\phi(x_1)$. I know that for the gaussian kernel $\phi$ is infinite dimensional and other properties of kernels but do not have an explicit representation for $\phi$.

I wish to know the explicit representation for $\phi$ of common kernels eg gaussian, periodic, matern etc.

(Links to papers or books would be great!)

$\endgroup$
2
$\begingroup$

Interesting. Why would you want to know that?

At least the "lifting" function of the polynomial kernel is well known (and on wikipedia): https://en.wikipedia.org/wiki/Polynomial_kernel

Two very good books on the topic:

  • Kernel Methods for Pattern Analysis, John Shawe-Taylor, Nello Cristianini
  • An Introduction to Support Vector Machines and Other Kernel-based Learning Methods, Nello Cristianini, John Shawe-Taylor

Since they are both from the same authors, they correlate somewhat. I would start with the first one if you are more into kernels than SVMs with kernels. It also has kernel expansions for other types than the polynomial kernel.

Btw: When used, the rbf kernel is normally not infinite-dimensional. The regulariation of the underlying modelling problem usually imposes finite dimensionality. Unfortunately this is how far it was explained to me. :/

$\endgroup$
  • $\begingroup$ Thanks for the pointers - very useful! I am trying to understand the relationship between the RKHS of different kernels and it seemed like a good place to start. $\endgroup$ – j__ Feb 12 '16 at 20:27
  • 1
    $\begingroup$ Be aware though. These RKHS are usually high-dimensional as you have already noticed. The human understanding of such high-dimensional spaces is usually completely off: people still expect clusters like they often see them in 2D but the matter of the fact is that we can not understand a high-dimensional RKHS like the one of the polynomial kernel. Instead thinking of clusters, it is often better to think of manifolds in the RKHS, i.e. your data only populates a sub-space of the RKHS. Maybe if you take this direction you will actually get the gist of it faster than heading straight for it. $\endgroup$ – pAt84 Feb 12 '16 at 20:45
  • $\begingroup$ thanks, I was thinking a little along that direction already. For examples, if we have 10 points we know they live in at most 10 dimensions and could we as such write these this subspace analytically (even as a infinite series). $\endgroup$ – j__ Feb 12 '16 at 20:50
  • 1
    $\begingroup$ If this were the case you might get there very easily with PCA, which can be performed in the RKHS as well, using a kernel. $\endgroup$ – pAt84 Feb 12 '16 at 22:03
  • $\begingroup$ that's a really interesting comment! Although I can't quite see how I would perform pca in the RKHS if it was say infinite dimensional. Any suggestions of how to approach this? $\endgroup$ – j__ Feb 12 '16 at 22:09
1
$\begingroup$

For anyone else who was interested in this I found a really nice paper that looks at the Gaussian kernel specifically (link here). Hope it helps someone in the future.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.