What is the posterior distribution $p(\textbf{f} | \textbf{y})$ for a Gaussian Process regression?

Suppose that $p(y_n |x_n, f) = N(f(x_n), \sigma^2)$ with prior on $\textbf{f} = [f(x_1), \ldots f(x_n)]^T$ as $p(\textbf{f}) = N(0, K)$ where $K = K(X, X)$ is the $n\times n$ Kernel matrix. How can I derive the expression for the posterior $f(\textbf{f} | \textbf{y})$?

It is not clear to me whether the posterior is again going to be a Gaussian Process or it is going to be a Gaussian. Please help.

Note: I am not asking about predictive posterior which every books seem to discuss. None of the books even mention the posterior I am looking for

  • $\begingroup$ Yes the posterior will be Gaussian, and you can use a combination of Bayes theorem and completing the square to show this. $\endgroup$ – microhaus Apr 10 at 14:29
  • $\begingroup$ @microhaus These notes say otherwise, Slide 4/6 mlg.eng.cam.ac.uk/teaching/4f13/1920/gp%20and%20data.pdf $\endgroup$ – Abhay Apr 10 at 14:33
  • 1
    $\begingroup$ Unless there is something I have missed, I am unable to understand how on slide 4/6, "Gaussian process prior with zero mean and covariance function $k$ and likelihood [in slides] leads to a Gaussian process posterior" amounts to saying otherwise? $\endgroup$ – microhaus Apr 10 at 14:37
  • $\begingroup$ Also, the predictive posterior that every book seems to discuss and the posterior you are looking for are the same. It is simply the conditional Gaussian distribution where $y_1,...,y_n$ are fixed at $x_1,...,x_n$ and we find posterior distribution (or conditional distribution) at any new $x_i, x_j, ...$. $\endgroup$ – Zeel B Patel May 6 at 17:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.