# Are Matérn class kernels universal kernels or not?

This is a question that I can't find the solution. I don't know it is a open question or it is a well-known result that can be attained from several lemmas.

Here are the definition of Matérn class kernel:

$M_{\sigma^2,\nu,\rho}(x,y)=\sigma^2\frac{2^{1-\nu}}{\Gamma(\nu)}(\sqrt{2\nu}\frac{||x-y||}{\rho})^{\nu}K_{\nu}(\sqrt{2\nu}\frac{||x-y||}{\rho})$

where $\sigma^2$ is a length scale parameter, $\Gamma(.)$ is gamma function, $K_{\nu}$ is the modified Bessel function of the second kind, and $\nu,\rho$ are non-negative parameters of covariance.

I use the definition of universal kernel in Universal Kernels(Charles A. Micchelli):

Given any prescribed compact subset Z of a Hausdorff topological space X, A kernel is universal if for any positive number $\epsilon$ and any function f ∈ C(Z) (continuous function on Z) there is a function g ∈ K(Z) (RKHS space generated by the kernel) such that sup norm $||f-g||_{Z}<\epsilon$

I know that frequently used Gaussian and Laplacian kernels are universal kernels and they are special case of Matérn class so I am wondering what kind of $\nu$ will lead to a universal kernel.

I am interesting in this problem because I am now studying Gaussian Process and Bayesian Optimization [Niranjan Srinivas,Jonathan Scarlett, Jasper Snoek] and I find that not only in theory they consider Matérn/Gaussian Kernel but also in real-world implementation. The default Kernel choice of many Bayesian optimization tools are Matérn 5/2. I don't think it is reasonable if it's not universal but I don't know how to prove it.

Yes, Matérn kernels are universal, for any parameters.

First note that $M_{\sigma^2,\nu,\rho}(x, y) = \Psi(x - y)$, i.e. the kernel is translation-invariant. It is also bounded and continuous.

The first main result we'll need is from:

Sriperumbudur, Fukumizu, and Lanckiret. Universality, Characteristic Kernels and RKHS Embedding of Measures. JMLR 12(Jul):2389−2410, 2011.

They show that the notion they call $cc$-universal is equivalent to the universality of Michhelli et al. (Remark 3). They also show that for bounded, continuous, translation-invariant kernels on $\mathbb R^d$, being characteristic implies being $cc$-universal (Figure 1 part 3).

It was previously shown that bounded continuous translation-invariant kernels on $\mathbb R^d$ are characteristic if and only if the Fourier transform of $\Psi$, which we know to be a nonnegative measure from Bochner's theorem, has support on all of $\mathbb R^d$: Theorem 7 of

Sriperumbudur, Gretton, Fukumizu, Lanckriet, and Schölkopf. Injective Hilbert Space Embeddings of Probability Measures. 21st Annual Conference on Learning Theory, 2008.

But the Fourier transform of $\Psi$ for the Matérn kernel, from equation (4.15) of Rasmussen and Williams, Gaussian Processes in Machine Learning (chapter 4 pdf), is the density $$S(s) = \sigma^2 \frac{2^d \pi^{d/2} \Gamma(\nu + d/2) (2 \nu)^\nu}{\Gamma(\nu) \rho^{2 \nu}} \left( \frac{2 \nu}{\rho^2} + 4 \pi^2 \lVert s \rVert^2 \right)^{-\nu + d/2}$$ where $d$ is the dimension. This of course has support on all of $\mathbb R^d$, so the Matérn kernel is characterestic, so it's $cc$-universal.

• Thank you so much for your self-contained and clear answer. To be honest, I do find Sriperumbudur's paper and find that Matern class is cc-universal. But I just read the introduction so I don't realize that they are equivalent. And I also have Gaussian Processes in Machine Learning at hand. So I guess the reason why we often use Matern 3/2 and Matern 5/2 is because that they have simple form and is 1/2-times differentiable. Jun 28, 2018 at 17:34
• I think that I need to study more math. If I want to understand the story between Gaussian Process and RKHS space, what reference would you recommend me to study? Jun 28, 2018 at 17:37
• GPML has a decent basic overview. I don't know of a good more-detail-than-that-but-not-mega-technical source; it's a significant gap in kernel methods right now imo.... Jun 28, 2018 at 19:55
• Could you further explain the last sentence, what is the current interest in kernel method right now? I am a CS master student. I am a layman of kernel method, I study it because I try to understand Bayesian Optimization and since they often use GP as a prior, kernel is everywhere. I only treat kernel as a tool to control smoothness (Kridging and kernel correspond to mean square differentibility) it will be good to know what kernel method specialist actually cares about right now. Jun 28, 2018 at 20:53
• Sorry, I didn't mean that it's a research gap, just that I'm not aware of a book that is detailed but not too hardcore mathy. Berlinet-Thomas/Agnan and Steinwart/Christmann are excellent detailed mathematical books about the topic. Jun 28, 2018 at 20:56