Multi-kernel Gaussian Process model?

Question

In Gaussian Process (GP), the kernel (covariance function) is used to measure the similarity between every two points. Currently I am using the Squared Exponential (SE) kernel to measure the similarity between graphs with all edge weights equal to 1. The SE kernel is based on the walking lengths. In theory, I can get a length-1 kernel, length-2 kernel, length-3 kernel and so on.

If I only pick one kernel, there will be a loss of information. So I am thinking if I can somehow combine all kernels together, for example, a linear combination, and the weights for each kernel can also be learned when training the GP model. Is this possible? Any suggestion is appreciated.

Answer 1

I am not sure if I understand you correctly. Yes, you can combine several kernels by adding and multiplying (cf. section 4.2.4 in Rasmussen/Williams, http://www.gaussianprocess.org/gpml/chapters/RW.pdf). More precisely, if $Y(x) \sim \mathcal{GP}(m, k)$ and $Z(x) \sim \mathcal{GP}(m', k')$ (mean and kernel function) are independent Gaussian processes, then $(Y+Z)(x)\sim \mathcal{GP}(m+m', k+k')$. For multiplication this is a little bit more involved, because the product of two Gaussian processes will not be a Gaussian process itself, but it will have mean $m m'$ and covariance $k k'$, and there will exist a Gaussian process with those (just it will not be equal to the product, unlike the summation case).

But this does not address 'loss of information' (and TBH I am not sure if I understand you there either), rather it corresponds to describing a decomposition of your process into components. I would encourage you to study the very instructive Mauna Loa example in section 5.4.3 in Rasmussen/Williams on this.

Regarding weights in $\alpha Y + \beta Z$, you can consider a constant $\alpha$ as a Gaussian process with a constant kernel (this is also supported e.g. by the sklearn GP toolbox), so yes, you can train those weights too. Again, I would point to the Mauna Loa example. (Correction: Kernel constructors in sklearn usually have a parameter for the scaling factor; however, you can also multiply with a ConstantKernel, it's equivalent. The point is that these weights behave like any other parameter of the kernel, no special treatment required in the parameter fitting)

EDIT: See also here for copyable code: https://scikit-learn.org/stable/auto_examples/gaussian_process/plot_gpr_co2.html