Are "kernel methods" and "reproducing kernel Hilbert spaces" related?

Question

Are "kernel methods" and "reproducing kernel Hilbert spaces" related?

Specifically, is the "kernel" used in the term "kernel methods" the same (type of) "kernel" as that used in the term "reproducing kernel Hilbert space"?

Note that I already checked the Wikipedia pages for the two topics, and there was no mention of "kernel method" on the page for "reproducing kernel Hilbert space" nor vice versa.

I want to find a textbook about reproducing kernel Hilbert spaces (see here), and this would be a lot easier if the two concepts were essentially the same, since "kernel methods" seems to be a topic which is covered frequently in machine learning textbooks.

On the other hand, if they are different, then I will have to expend some effort in making sure that I understand the difference between the two in order to avoid getting confused.

Answer 1

The Wikipedia pages for "Kernel Method" and "reproducing kernel Hilbert space" both refer to Mercer's theorem, which is the connection. If the kernel used in a kernel method is a "Mercer kernel" (that is, it satisfies the Mercer condition), then the method works "as if" it was operating in the Hilbert space (a function space) corresponding to the kernel. This is called the kernel trick.

Note that kernel methods can be applied with a non-Mercer kernel however. To quote the Wikipedia page: "Empirically, for machine learning heuristics, choices of a function $k$ that do not satisfy Mercer's condition may still perform reasonably if $k$ at least approximates the intuitive idea of similarity."