3
$\begingroup$

I'm a little bit confused about the posterior distribution for Gaussian processes. If we consider the case with noise-free data, we have a prior $f|X \sim N(0, K)$. If we are given $\{(x_i, f_i): i=1.,\ldots, n\}.$ The joint distribution of training outputs $f$ and test outputs $f_*$ according to prior is $$\begin{bmatrix} f \\ f_* \end{bmatrix} \sim N\left(0, \begin{bmatrix} K(X,X) & K(X,X_*) \\ K(X_*,X) & K(X_*,X_*) \end{bmatrix}\right).$$

I think the predictive distribution is: $$f_*|X_*,X,f \sim N\left( K(X_*, X)K(X,X)^{-1}f, K(X_*,X_*)-K(X_*,X)K(X,X)^{-1}K(X,X_*) \right).$$

If this is correct, then what is the posterior distribution? I may be getting confused by the fact that the GP is "parameter-free"... I'm used to the prior being a distribution over the parameters, but in this case there are none... Also, since the prior is over functions $f$, I guess the posterior should be as well?

Improve this question
$\endgroup$
3
  • 3
    $\begingroup$ Where did you read that GP is parameter-free? I suspect they mean the parameters of the linear model which are the coefficients that linearly combine the basis functions. $\endgroup$
    – Leila
    Commented Jul 5, 2017 at 10:45
  • $\begingroup$ Some discussion here: stats.stackexchange.com/questions/46588/… $\endgroup$
    – theQman
    Commented Jul 5, 2017 at 13:42
  • 1
    $\begingroup$ It is called "nonparametric" not "parameter-free". Here I explained what it means: stats.stackexchange.com/a/254960/66491 $\endgroup$
    – Leila
    Commented Jul 6, 2017 at 11:12

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.