I want to do a full Bayesian analysis of a Gaussian process regression problem: $$x\sim{GP[\mu,K(t_i,t_j)]},$$ where $\mu$ and $K$ are mean and covariance functions and $$K(t_i,t_j)=\sigma_f^2\exp\left(-\frac{(t_i-t_j)^2}{2l^2}\right)+\sigma_n^2\delta(t_i,t_j),$$ where $\sigma_f^2$,$l$, and $\sigma_n^2$ are parameters of the covariance function. It seems like I can select a noninformative prior density on $\mu$ to be equal to $1$. How do I select noninformative hyperpriors for $\sigma_f^2$,$l$, and $\sigma_n^2$? Any reference will be very helpful.


There is no such thing as a truly noninformative prior--even the prior you've selected for the mean is not uniform under a reparameterisation.

Having said that, you could use proper uniform priors on the covariance and noise parameters, for example with a lower limit of 0 and an upper limit of $U$. Then you could vary $U$ to ensure that the arbitrary upper limit is not unduly affecting your results.

However, I would generally advocate using an informative prior... but that is another answer.

  • $\begingroup$ Thanks for the suggestion! I have one more question though. Can we say that noninformative prior means the prior with minimum information? If so, can we find out an analytical expression for noninformative prior in this case similar to Jeffreys prior? $\endgroup$ Nov 17 '17 at 20:36
  • 1
    $\begingroup$ You are alluding to maximum entropy priors. Even in this case you have to input some information (the expected value conditions). $\endgroup$ Nov 17 '17 at 21:01
  • $\begingroup$ Can we differentiate between Maximum entropy and noninformative prior? $\endgroup$ Nov 17 '17 at 21:21
  • 1
    $\begingroup$ Noninformative does not really exist. There are various approaches that try to be "objective". I suggest you take a look at other questions to help answer this or ask a new question. $\endgroup$ Nov 17 '17 at 21:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.