2
$\begingroup$

In Rasmussen's Gaussian Processes for Machine Learning, the joint distribution of noisy function observations, $y=f(x)+\epsilon$, at $x$ and noiseless function evaluations, $f^\star$, at unseen points, $x^\star$, given the hyperparameters of the kernel and the noise variance is

$$ \begin{bmatrix} y \\ f^\star \end{bmatrix}\sim N\biggr(0, \begin{bmatrix} K(x,x)+\sigma^2I & K(x,x^\star)\\ K(x,x^\star) & K(x^\star,x^\star) \end{bmatrix}\biggr ) $$

Rasmussen uses the above to derive $p(f^\star|y,X,X^\star)$, which Rasmussen refers to as the predictive distribution. He appears to construct $95\%$ confidence regions by taking the pointwise mean and 1.96 times the standard deviation given by the conditional distribution of $f^\star$. What is the interpretation of these pointwise intervals around the mean and its relationship to $f$?

I believe the predictive distribution above is a distribution over functions which describes the set of plausible values $f^\star$ could take on if I sampled from the conditional. It isn't clear to me though that these intervals are directly related to the realized values of $f$ which occur in the training points. Is that accurate?

$\endgroup$
1
  • 1
    $\begingroup$ What is "Rasmussen"? Some paper from 1872? A highly cited paper from 2003, "Gaussian Processes for Machine Learning"? The book with same title, by Rasmussen and Williams? You could give a link, and the publication year (just once). $\endgroup$
    – Ute
    May 30, 2023 at 22:17

1 Answer 1

1
$\begingroup$

You can think of it like this: a function is a mapping $f: x \to y$. We use Gaussian Processes to model random functions $f \sim \mathcal{GP}$, where the mapping is non-deterministic. GP takes some points $x$ and the realizations of the functions $f(x) = y$ to learn the random function $f$. Using the symbols you used in your question $x$ are the observed inputs, and $x_*$ are the new, unobserved ones. The intervals describe the distribution over functions (as modeled with GP). We can evaluate the GP over arbitrary inputs $x_*$ to get the draws from the mappings for those inputs. So "the points" are just the mappings that we happened to look at, but GP can be evaluated for any input $x_*$ and tell us what is the distribution of the realizations of $f$ at those points.

$\endgroup$
2
  • $\begingroup$ Okay, got it. The intervals outline the distribution of the realizations of $f$ at the unobserved points. The GP defines a non-deterministic mapping from $x\rightarrow y$, but once I sample from the GP, that function is fixed? Would it be wrong to say that? $\endgroup$ Jun 1, 2023 at 12:50
  • $\begingroup$ @user1848065 the sample would be a realization of the random function, the same as running rnorm() in R would produce a sample that is a realization of the Gaussian random variable. $\endgroup$
    – Tim
    Jun 1, 2023 at 13:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.