2
$\begingroup$

The book Gaussian Processes for Machine Learning (GPML) by Rasmussen and Williams (2006) provides a graphical model for GP regression but does not explain it in great detail, so I have a few questions about it: Graphical model for a GP

  1. Is $c$ the number of context points, hence the $c$ subscript for $y_c, f_c, x_c$?

  2. Does "Gaussian field" refer to the fact that all of the (infinite) function evaluations $f_i$ are jointly Gaussian?

  3. Does this recreated and simplified graphical model also make sense, or is there something wrong about it that I'm missing?

    enter image description here

Improve this question
$\endgroup$
2
$\begingroup$
  1. Yes, 'c' represents the size of the training data.
  2. No, a stochastic process like the Gaussian process is a special case of a random field where the stochastic process takes values in Euclidean space. The usage of the term "Gaussian field" refers to the relaxation of constraints that come along with random fields(please refer wiki for more info on this), which allow taking values not restricted to only the euclidean space. The existence of a Gaussian field instead of a Gaussian process is to facilitate manifold learning, which requires manipulations in imaginary space along with real space(hence complex space).
  3. It makes sense, as long as the shaded regions indicate observed values and the unshaded ones indicate unobserved values.

Hope that helps.:)

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks for the answer! So could we label the fully-connected latent variables $f_i$ as "Gaussian process" instead of "Gaussian field" to make the diagram less complicated, or would that no longer be accurate? $\endgroup$ – Christabella Irwanto Feb 16 '20 at 13:33
  • 1
    $\begingroup$ Yes, you could do that; it would mean that you have imposed restriction on the modeling potential since now you would only be taking values in the euclidean space instead of the euclidean+imaginary space. $\endgroup$ – Nizam Feb 16 '20 at 15:39
  • 1
    $\begingroup$ Also, may I suggest you use the standard notation from the book if you intend to publish in peer-reviewed journals since it is the de-facto source when it comes to GPs, else you are good to go with the simpler version, which is a valid subset of the fully-qualified model. Hope this helps. $\endgroup$ – Nizam Feb 16 '20 at 15:52
  • $\begingroup$ Thank you very much! In the GPML book however the figure caption describes the diagram as a "GP" and doesn't explain "Gaussian field" anywhere else, so I thought it was better to trim unnecessary information if my content only relates to GPs and not Gaussian fields. (Especially since I am unequipped to fully understand how Gaussian fields generalize GPs to complex space, and am having trouble finding accessible learning resources. "Wikipedia" states that "a one-dimensional Gaussian random field is a GP" but without further explanation.) $\endgroup$ – Christabella Irwanto Feb 17 '20 at 10:13
  • 1
    $\begingroup$ You're welcome. Yes, as I said earlier, there is no harm in considering GPs as a special case of Gaussian Fields. I cannot remember any other treatise at the moment treating gaussian processes. But I can refer you to the lecture series by Neil Lawrence youtube.com/watch?v=ewJ3AxKclOg and that by Zoubin ghahramani youtube.com/watch?v=IEpc2ClaYH8. $\endgroup$ – Nizam Feb 17 '20 at 10:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.