# What are the various basic kernels available?

I am currently following the book Gaussian Processes for Machine Learning by C.E. Rasmussen and C.K.I. Williams and I have come across various kernels in their Chapter 4

I have also gone through the kernel cookbook which is a nice description of various basic kernels available and how we can create new kernels from adding, multiplying, and convoluting these basic kernels.

Since we know that regression in a Gaussian process hinges on the choice of kernels, I want to ask if there are basic kernels other than the following and what kind of functions do they draw?

1. Constant : $\sigma _{o}^{2}$

2. Linear kernel : $k_{\textrm{Lin}}(x, x') = (x - c)(x' - c)$

3. Squared exponential kernel : $k_{\textrm{SE}}(x, x') = \sigma^2\exp\left(-\frac{(x - x')^2}{2\ell^2}\right)$ (subset of 7)

4. Matern : $\frac{1}{{2}^{\nu -1}\Gamma (\nu )}{\left( \frac{\sqrt{2\nu }}{l}r\right)}^{\nu }{K}_{\nu }\left( \frac{\sqrt{2\nu }}{l}r\right)$

5. Rational quadratic kernel : $k_{\textrm{RQ}}(x, x') = \sigma^2 \left( 1 + \frac{(x - x')^2}{2 \alpha \ell^2} \right)^{-\alpha}$ (subset of 4)

6. Exponential : $\exp\left( - \frac{r}{l}\right)$ (subset of 7)

7. $\gamma$ - Exponential : $\exp\left( -{\left(\frac{r}{l}\right)}^{\gamma }\right)$

8. Periodic kernel : $k_{\textrm{Per}}(x, x') = \sigma^2\exp\left(-\frac{2\sin^2(\pi|x - x'|/p)}{\ell^2}\right)$

9. Neural networks

• Neural networks are not a kernel. Dec 18, 2014 at 17:20
• @MarcClaesen I suppose the question refers to the so-called neural network covariance function (see, e.g., the linked chapter 4 of the Rasmussen & Williams book. Dec 18, 2014 at 17:56
• @JuhoKokkala Yes I was refering to the neural network covariance function. Dec 18, 2014 at 20:17

Neural networks are not a kernel, they are a learning algorithm.

Plenty of kernel functions exist, such as:

• sigmoid, popular in the early days of kernel methods due to their influence in neural networks; not really used heavily now
• Tanimoto/Jaccard/diffusion, popular for binary features
• tree/graph kernels, popular in natural language processing
• histogram kernel, popular in image processing -- essentially it's a very fast approximation to the RBF kernel

The right kernel depends very much on the nature of the data. Often the best kernel is a custom-made one, particularly in bioinformatics. The Gaussian/RBF and linear kernels are by far the most popular ones, followed by the polynomial one.

• Thanks, as mentioned above I was refering to the neural network covariance/kernel as mentioned in the above book. Also I wanted this question to become some sort of a repository of all the different basic kernels we have/know so far. Dec 18, 2014 at 20:19
• @AnkitChiplunkar unfortunately, questions regarding enumerations like this one are regularly being closed on cross-validated. Dec 18, 2014 at 21:45