I'm a little bit confused about the posterior distribution for Gaussian processes. If we consider the case with noise-free data, we have a prior $f|X \sim N(0, K)$. If we are given $\{(x_i, f_i): i=1.,\ldots, n\}.$ The joint distribution of training outputs $f$ and test outputs $f_*$ according to prior is $$\begin{bmatrix} f \\ f_* \end{bmatrix} \sim N\left(0, \begin{bmatrix} K(X,X) & K(X,X_*) \\ K(X_*,X) & K(X_*,X_*) \end{bmatrix}\right).$$

I think the predictive distribution is: $$f_*|X_*,X,f \sim N\left( K(X_*, X)K(X,X)^{-1}f, K(X_*,X_*)-K(X_*,X)K(X,X)^{-1}K(X,X_*) \right).$$

If this is correct, then what is the posterior distribution? I may be getting confused by the fact that the GP is "parameter-free"... I'm used to the prior being a distribution over the parameters, but in this case there are none... Also, since the prior is over functions $f$, I guess the posterior should be as well?

  • 3
    $\begingroup$ Where did you read that GP is parameter-free? I suspect they mean the parameters of the linear model which are the coefficients that linearly combine the basis functions. $\endgroup$
    – Seeda
    Jul 5, 2017 at 10:45
  • $\begingroup$ Some discussion here: stats.stackexchange.com/questions/46588/… $\endgroup$
    – theQman
    Jul 5, 2017 at 13:42
  • 1
    $\begingroup$ It is called "nonparametric" not "parameter-free". Here I explained what it means: stats.stackexchange.com/a/254960/66491 $\endgroup$
    – Seeda
    Jul 6, 2017 at 11:12


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.