1
$\begingroup$

In Gaussian Process (GP), the kernel (covariance function) is used to measure the similarity between every two points. Currently I am using the Squared Exponential (SE) kernel to measure the similarity between graphs with all edge weights equal to 1. The SE kernel is based on the walking lengths. In theory, I can get a length-1 kernel, length-2 kernel, length-3 kernel and so on.

If I only pick one kernel, there will be a loss of information. So I am thinking if I can somehow combine all kernels together, for example, a linear combination, and the weights for each kernel can also be learned when training the GP model. Is this possible? Any suggestion is appreciated.

$\endgroup$
1
  • $\begingroup$ Using one kernel is already computationally expensive, using more would be more expensive. You have enough time & compute for that? If your data is very small it’s probably not a problem, but will be a problem fast as the data grows. $\endgroup$
    – Tim
    Jul 25 at 20:48
3
$\begingroup$

I am not sure if I understand you correctly. Yes, you can combine several kernels by adding and multiplying (cf. section 4.2.4 in Rasmussen/Williams, http://www.gaussianprocess.org/gpml/chapters/RW.pdf). More precisely, if $Y(x) \sim \mathcal{GP}(m, k)$ and $Z(x) \sim \mathcal{GP}(m', k')$ (mean and kernel function) are independent Gaussian processes, then $(Y+Z)(x)\sim \mathcal{GP}(m+m', k+k')$. For multiplication this is a little bit more involved, because the product of two Gaussian processes will not be a Gaussian process itself, but it will have mean $m m'$ and covariance $k k'$, and there will exist a Gaussian process with those (just it will not be equal to the product, unlike the summation case).

But this does not address 'loss of information' (and TBH I am not sure if I understand you there either), rather it corresponds to describing a decomposition of your process into components. I would encourage you to study the very instructive Mauna Loa example in section 5.4.3 in Rasmussen/Williams on this.

Regarding weights in $\alpha Y + \beta Z$, you can consider a constant $\alpha$ as a Gaussian process with a constant kernel (this is also supported e.g. by the sklearn GP toolbox), so yes, you can train those weights too. Again, I would point to the Mauna Loa example. (Correction: Kernel constructors in sklearn usually have a parameter for the scaling factor; however, you can also multiply with a ConstantKernel, it's equivalent. The point is that these weights behave like any other parameter of the kernel, no special treatment required in the parameter fitting)

EDIT: See also here for copyable code: https://scikit-learn.org/stable/auto_examples/gaussian_process/plot_gpr_co2.html

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.