Let the prior on the regression function $f(·)$ be a GP, denoted by $f(x) ∼ GP (m(x), κ(x, x'))$. $m(x)$ is usually $0$, so let us consider that too.

From what I understand a Gaussian Process can be seen as a prior over a set of functions.

I have a set of observations $\{(x_1,y_1), (x_2,y_2), ..., (x_N,y_N)\}$ and I want to predict $y^*$ for my specific $x^*$. I constrain the GP on my observations and then I predict $y^* = f(x^*)$.

My question is what happens to my mean and covariance $m(x), κ(x, x')$? I initially gave them a value, but did that change when I constrained the GP? In another words, is $m(x), κ(x, x')$ different from the values I set initially?


The mean and the covariance are functions. The GP prior is a function over the space determined by your mean and covariance functions. Then when you have data your space of functions is reduced to ones that fit your data. So the mean and covariance you didn't give any values, you made them potentially infinite in size. This is why its called a non parametric model.

| cite | improve this answer | |
  • $\begingroup$ So can I get a formula dependent on the original $m(x)$ and on my data to get the new posterior mean? I understand that this is also a function. $\endgroup$ – Catarina Alves Feb 19 at 21:40
  • $\begingroup$ In a regression with a Gaussian likelihood your posterior mean is $ E[f^*|X,y,X^*] = K(X^*,X)[K(X,X) + \sigma^2 \mathbf(I)]^-1(\mathbf{y} - m(X)) + m(X^*) $. So the only elements that change if you want to predict at new data points is the cross kernel function and the mean function for the test data points. Think of that unchanged term as like the real computation of a GP regression. So yeah you do get a formula that is dependent your data as well as the functions that you defined for your original training data set. $\endgroup$ – Chango Feb 20 at 8:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.