I thought functional analysis was long thought to be old fashioned and generally a dead research area.

It seems that all of a sudden there is a huge fascination with so-called reproducing kernel Hilbert space in the machine learning community. Specifically, with some applications of the Mercer's theorem.

A Hilbert space is a complete vector space equipped with an inner product. Nearly all of machine learning works with the Hilbert space $(\mathbb{R}^n, \langle, \rangle)$ already. So I don't see the point of looking into this particular one.

Can anyone provide a simple application that illustrates why it needs to be done within this reproducing kernel Hilbert space setup?

  • 3
    $\begingroup$ Functional analysis is a fundamental tool in quantum mechanics. I highly doubt it's a dead research area. $\endgroup$
    – mesllo
    Commented Nov 10, 2021 at 1:58

1 Answer 1


The typical way to give some intuition for reproducing kernel spaces (and, in particular, the kernel trick), is the application area of support vector machines. The aim is to linearly separate two classes of points in $\mathbb R^n$, which works fine if they actually are linearly separable.

If they are not, the kernel trick provides (in certain situations) a possibility to transform the data points into another space, the so-called reproducing kernel Hilbert space or feature space, where the transformed points become linearly separable.

A good description can be found here.

Of course, this is just one of hundreds of applications of the kernel trick (or RKHS in general), but it is one which hopefully clarifies its power and justifies its usefulness.

  • 2
    $\begingroup$ Is separability only important for classification tasks? If this property is valuable in regression tasks, what's the intuition behind how RKHS makes regression problems tractable? $\endgroup$ Commented Jun 28, 2021 at 16:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.