# Pointwise Confidence Regions in Gaussian Process

In Rasmussen's Gaussian Processes for Machine Learning, the joint distribution of noisy function observations, $$y=f(x)+\epsilon$$, at $$x$$ and noiseless function evaluations, $$f^\star$$, at unseen points, $$x^\star$$, given the hyperparameters of the kernel and the noise variance is

$$\begin{bmatrix} y \\ f^\star \end{bmatrix}\sim N\biggr(0, \begin{bmatrix} K(x,x)+\sigma^2I & K(x,x^\star)\\ K(x,x^\star) & K(x^\star,x^\star) \end{bmatrix}\biggr )$$

Rasmussen uses the above to derive $$p(f^\star|y,X,X^\star)$$, which Rasmussen refers to as the predictive distribution. He appears to construct $$95\%$$ confidence regions by taking the pointwise mean and 1.96 times the standard deviation given by the conditional distribution of $$f^\star$$. What is the interpretation of these pointwise intervals around the mean and its relationship to $$f$$?

I believe the predictive distribution above is a distribution over functions which describes the set of plausible values $$f^\star$$ could take on if I sampled from the conditional. It isn't clear to me though that these intervals are directly related to the realized values of $$f$$ which occur in the training points. Is that accurate?

• What is "Rasmussen"? Some paper from 1872? A highly cited paper from 2003, "Gaussian Processes for Machine Learning"? The book with same title, by Rasmussen and Williams? You could give a link, and the publication year (just once).
– Ute
May 30, 2023 at 22:17

You can think of it like this: a function is a mapping $$f: x \to y$$. We use Gaussian Processes to model random functions $$f \sim \mathcal{GP}$$, where the mapping is non-deterministic. GP takes some points $$x$$ and the realizations of the functions $$f(x) = y$$ to learn the random function $$f$$. Using the symbols you used in your question $$x$$ are the observed inputs, and $$x_*$$ are the new, unobserved ones. The intervals describe the distribution over functions (as modeled with GP). We can evaluate the GP over arbitrary inputs $$x_*$$ to get the draws from the mappings for those inputs. So "the points" are just the mappings that we happened to look at, but GP can be evaluated for any input $$x_*$$ and tell us what is the distribution of the realizations of $$f$$ at those points.
• Okay, got it. The intervals outline the distribution of the realizations of $f$ at the unobserved points. The GP defines a non-deterministic mapping from $x\rightarrow y$, but once I sample from the GP, that function is fixed? Would it be wrong to say that? Jun 1, 2023 at 12:50
• @user1848065 the sample would be a realization of the random function, the same as running rnorm() in R would produce a sample that is a realization of the Gaussian random variable.