3
$\begingroup$

A kernel, $k(x_1, x_2)$, has the interesting property that it may be represented as the dot product in a reproducing kernel hilbert space (RKHS), $\phi(x_0)\phi(x_1)$. I know that for the gaussian kernel $\phi$ is infinite dimensional and other properties of kernels but do not have an explicit representation for $\phi$.

I wish to know the explicit representation for $\phi$ of common kernels eg gaussian, periodic, matern etc.

(Links to papers or books would be great!)

Improve this question
$\endgroup$

2 Answers 2

Reset to default
3
$\begingroup$

Interesting. Why would you want to know that?

At least the "lifting" function of the polynomial kernel is well known (and on wikipedia): https://en.wikipedia.org/wiki/Polynomial_kernel

Two very good books on the topic:

  • Kernel Methods for Pattern Analysis, John Shawe-Taylor, Nello Cristianini
  • An Introduction to Support Vector Machines and Other Kernel-based Learning Methods, Nello Cristianini, John Shawe-Taylor

Since they are both from the same authors, they correlate somewhat. I would start with the first one if you are more into kernels than SVMs with kernels. It also has kernel expansions for other types than the polynomial kernel.

Btw: When used, the rbf kernel is normally not infinite-dimensional. The regulariation of the underlying modelling problem usually imposes finite dimensionality. Unfortunately this is how far it was explained to me. :/

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thanks for the pointers - very useful! I am trying to understand the relationship between the RKHS of different kernels and it seemed like a good place to start. $\endgroup$
    – j__
    Commented Feb 12, 2016 at 20:27
  • 1
    $\begingroup$ Be aware though. These RKHS are usually high-dimensional as you have already noticed. The human understanding of such high-dimensional spaces is usually completely off: people still expect clusters like they often see them in 2D but the matter of the fact is that we can not understand a high-dimensional RKHS like the one of the polynomial kernel. Instead thinking of clusters, it is often better to think of manifolds in the RKHS, i.e. your data only populates a sub-space of the RKHS. Maybe if you take this direction you will actually get the gist of it faster than heading straight for it. $\endgroup$
    – pAt84
    Commented Feb 12, 2016 at 20:45
  • $\begingroup$ thanks, I was thinking a little along that direction already. For examples, if we have 10 points we know they live in at most 10 dimensions and could we as such write these this subspace analytically (even as a infinite series). $\endgroup$
    – j__
    Commented Feb 12, 2016 at 20:50
  • 1
    $\begingroup$ If this were the case you might get there very easily with PCA, which can be performed in the RKHS as well, using a kernel. $\endgroup$
    – pAt84
    Commented Feb 12, 2016 at 22:03
  • $\begingroup$ that's a really interesting comment! Although I can't quite see how I would perform pca in the RKHS if it was say infinite dimensional. Any suggestions of how to approach this? $\endgroup$
    – j__
    Commented Feb 12, 2016 at 22:09
3
$\begingroup$

For anyone else who was interested in this, there is a paper by Xu et al (2006) that is good. I hope it might help someone in the future.

In case the DOI link above breaks, the paper is called

J. Xu et al, "An explicit construction of A reproducing gaussian kernel hilbert space," in 2006, . DOI: 10.1109/ICASSP.2006.1661340.

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.