2
$\begingroup$

I'm studying Gaussian processes and currently reading the standard reference Gaussian Processes for Machine Learning. However, so far I didn't see any example of a multivariate Gaussian process nor did I find a reference in the Index. So I'm looking for a good source for the (theoretical) treatment with application of multivariate Gaussian Processes.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ A multivariate GP is actually just another GP! All you need to do is define a cross-covariance function to understand how the different variables correlate. And then, as usual, define a covariance function for each individual variable $\endgroup$
    – jcken
    Commented Sep 11, 2020 at 19:33
  • $\begingroup$ @jcken Thanks for your answer. I see, this makes sense, but do you know any references, even lecture notes which cover this formally? $\endgroup$
    – math
    Commented Sep 12, 2020 at 18:09
  • $\begingroup$ this paper might be a good start, or maybe references within $\endgroup$
    – jcken
    Commented Sep 12, 2020 at 18:18

1 Answer 1

Reset to default
2
$\begingroup$

Gaussian process models with multiple outputs are discussed briefly in section 9.1 of the GPML book (Rasmussen & Williams 2006). To paraphrase, some approaches include:

  1. Treat the outputs as independent, which simply involves fitting a separate model for each. This may be suboptimal because it fails to account for dependence between outputs.

  2. Model the underlying outputs as a priori independent, but with correlated noise. This induces correlations between outputs in the posterior.

  3. Use a covariance function that describes not only the correlation structure of each output, but also correlations between outputs (Cressie 1993). This is called 'cokriging' in the geostatistics literature.

  4. Model the outputs as linear combinations of multiple latent processes (Teh et al. 2005, Micchelli and Pontil 2005). A related approach involves treating the outputs as different convolutions of the same underlying white noise process (Boyle and Frean 2005).

References

  • Boyle, P. and Frean, M. (2005). Dependent Gaussian Processes. In Saul, L. K., Weiss, Y., and Bottou, L., editors, Advances in Neural Information Processing Systems 17, pages 217–224. MIT Press.

  • Cressie, N. A. C. (1993). Statistics for Spatial Data. Wiley, New York.

  • Micchelli, C. A. and Pontil, M. (2005). Kernels for Multi-task Learning. In Saul, L. K., Weiss, Y., and Bottou, L., editors, Advances in Neural Information Processing Systems 17, pages 921–928. MIT Press.

  • Teh, Y. W., Seeger, M., and Jordan, M. I. (2005). Semiparametric Latent Factor Models. In Cowell, R. G. and Ghahramani, Z., editors, Proceedings of Tenth International Workshop on Artificial Intelligence and Statistics, pages 333–340. Society for Artificial Intelligence and Statistics.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.