# Multi-kernel Gaussian Process model?

In Gaussian Process (GP), the kernel (covariance function) is used to measure the similarity between every two points. Currently I am using the Squared Exponential (SE) kernel to measure the similarity between graphs with all edge weights equal to 1. The SE kernel is based on the walking lengths. In theory, I can get a length-1 kernel, length-2 kernel, length-3 kernel and so on.

If I only pick one kernel, there will be a loss of information. So I am thinking if I can somehow combine all kernels together, for example, a linear combination, and the weights for each kernel can also be learned when training the GP model. Is this possible? Any suggestion is appreciated.

• Using one kernel is already computationally expensive, using more would be more expensive. You have enough time & compute for that? If your data is very small it’s probably not a problem, but will be a problem fast as the data grows.
– Tim
Jul 25, 2021 at 20:48

I am not sure if I understand you correctly. Yes, you can combine several kernels by adding and multiplying (cf. section 4.2.4 in Rasmussen/Williams, http://www.gaussianprocess.org/gpml/chapters/RW.pdf). More precisely, if $$Y(x) \sim \mathcal{GP}(m, k)$$ and $$Z(x) \sim \mathcal{GP}(m', k')$$ (mean and kernel function) are independent Gaussian processes, then $$(Y+Z)(x)\sim \mathcal{GP}(m+m', k+k')$$. For multiplication this is a little bit more involved, because the product of two Gaussian processes will not be a Gaussian process itself, but it will have mean $$m m'$$ and covariance $$k k'$$, and there will exist a Gaussian process with those (just it will not be equal to the product, unlike the summation case).
Regarding weights in $$\alpha Y + \beta Z$$, you can consider a constant $$\alpha$$ as a Gaussian process with a constant kernel (this is also supported e.g. by the sklearn GP toolbox), so yes, you can train those weights too. Again, I would point to the Mauna Loa example. (Correction: Kernel constructors in sklearn usually have a parameter for the scaling factor; however, you can also multiply with a ConstantKernel, it's equivalent. The point is that these weights behave like any other parameter of the kernel, no special treatment required in the parameter fitting)