Can someone provide a brief explanation as to why reproducing kernel Hilbert space is so popular in machine learning?

Question

I thought functional analysis was long thought to be old fashioned and generally a dead research area.

It seems that all of a sudden there is a huge fascination with so-called reproducing kernel Hilbert space in the machine learning community. Specifically, with some applications of the Mercer's theorem.

A Hilbert space is a complete vector space equipped with an inner product. Nearly all of machine learning works with the Hilbert space $(\mathbb{R}^n, \langle, \rangle)$ already. So I don't see the point of looking into this particular one.

Can anyone provide a simple application that illustrates why it needs to be done within this reproducing kernel Hilbert space setup?

Answer 1

The typical way to give some intuition for reproducing kernel spaces (and, in particular, the kernel trick), is the application area of support vector machines. The aim is to linearly separate two classes of points in $\mathbb R^n$, which works fine if they actually are linearly separable.

If they are not, the kernel trick provides (in certain situations) a possibility to transform the data points into another space, the so-called reproducing kernel Hilbert space or feature space, where the transformed points become linearly separable.

A good description can be found here.

Of course, this is just one of hundreds of applications of the kernel trick (or RKHS in general), but it is one which hopefully clarifies its power and justifies its usefulness.