I have the following problem. I'm reading through the Gaussian Process book http://www.gaussianprocess.org/gpml/chapters/RW2.pdf. In the bayesian linear regression it is suggested to use the Gaussian prior over the parameters. For the Gaussian process regression we also don't know the distribution of the process, it can be not gaussian. Can GPR always work well? I just don't understand why should we use gaussian distribution? Is there any paper or book?
$\begingroup$
$\endgroup$
0
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
I can't exactly define what you ask in this question. However, I have two hypothesis.
Why do we use Gaussian prior for Bayesian linear regression?
We use this prior as convenient one and one that has nice interpretation. Really, it is a quadratic penalty for parameters values.
Why do we use Gaussian process as a model for the data?
Realizations of Gaussian processes with a proper covariance function can provide nearly all functions we can encounter in "real life". Also, they are convenient and provide exact inference and marginal distribution.
$\begingroup$
$\endgroup$
If I understood your question correctly, it is about conjugate priors. This is the integral that you need to calculate to incorporate your prior into the likelihood function.
For a given likelihood function prob(X|Θ), a prior prob(Θ) is called a conjugate prior if the posterior prob(Θ|X) has the same algebraic form as the prior. If we have a choice regarding how we express our prior beliefs, we must use that form which allows us to carry out the integration shown above.
For a given algebraic form for the likelihood, the different forms for the prior prob (Θ) pose different levels of difficulty for the determination of the marginal in the denominator and,therefore, for the determination of the posterior.
The conjugate prior of a Gaussian distribution is Gaussian. Similarly, the conjugate prior of a binomial distribution is beta and the conjugate prior of a multinomial distribution is Dirichlet.
In short, the selection of the Gaussian to represent your prior is due to mathematical convenience.
