4
$\begingroup$

Assume that I have a gaussian regression problem, where I have a covariance function K that is estimated based on two kernels K1 and K2. For example K1 is a squared exponential kernel, and K2 is a periodic kernel. How can I know the contribution of each kernel on the prediction of the gaussian process?

The solution to this question is presented in Figures 5.7 and 5.8 in:

C. E. Rasmussen & C. K. I. Williams, Gaussian Processes for Machine Learning, the MIT Press, 2006, ISBN 026218253X. c 2006 Massachusetts Institute of Technology. (free online)

but the authors didn't explain how did they come up with these figures.

This is not a programming question nor a question about the book's specific example. I am interested in learning how do I estimate the influence of each component of the covariance function K on the prediction since it is extremely helpful in the inference, understanding, and interpretation of the gaussian process regression model.

Improve this question
$\endgroup$
7
  • $\begingroup$ Since the covariance kernel $k$ is the sum of four kernel $k_i$, the series $y(x)$ can be regarded as the sum of four independent unobserved series $y_i(x)$ with kernel $k_i$. A smoothed version $\widehat{y}_i$ of each component can be computed as a conditional expectation. My guess is that fig. 5.8 shows contours of $\widehat{y}_2(x)$ using the month and the year part of time $x$ as spatial coordinates. $\endgroup$
    – Yves
    Commented Apr 19, 2017 at 6:29
  • $\begingroup$ How do I compute the conditional expectation? $\endgroup$
    – Mahshrp
    Commented Apr 28, 2017 at 11:52
  • 1
    $\begingroup$ Use the classical formula for the computational expectation for a multivariate normal. The vector of the $n$ estimated values for component $i$ is given by $\widehat{\mathbf{y}}_i = \boldsymbol{\mu}_i + \mathbf{K}_i \mathbf{K}^{-1} [\mathbf{y} - \boldsymbol{\mu}]$. Here the elements $n \times n$ matrices $\mathbf{K}$ contain the covariance kernel values and the vectors $\boldsymbol{\mu}$ are expectations. $\endgroup$
    – Yves
    Commented Apr 29, 2017 at 13:55
  • $\begingroup$ Oops "computational" expecation is obviously conditional expectation. Sorry. $\endgroup$
    – Yves
    Commented Apr 29, 2017 at 15:35
  • 1
    $\begingroup$ I am currently trying to answer the same question and am making progress. Have a look at the following ticket plus follow the links like the CO2 data of Mauna Loa as given by Gaussian Processes for Machine Learning in chapter 5: github.com/GPflow/GPflow/issues/491 $\endgroup$
    – cs224
    Commented Aug 25, 2017 at 9:16

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.