# Hyperparameters in Gaussian process

Without going to far into the details, I am using a Gaussian Process for the prediction of the posterior given by:

$$p(\mathbf{T}\vert\mathbf{X},\boldsymbol{\theta}) = \int p(\mathbf{T}\vert f)p(f\vert \mathbf{X},\boldsymbol{\theta})df$$

Above I am just performing a marginalization to merge the uncertainties in the likelihood and the prior, but I have an issue, the hyperparameters $$\boldsymbol{\theta}$$, what kind of condition do we have on these after the integral has been made? I have always considered these as parameters that can be rather tricky to learn and is more to be chosen with great care.

I mean, I have a hard time interpreting the question I'm supposed to answer, please, correct me if I'm wrong, but I suppose my goal is to choose hyperparameters in such a way that I maximize my posterior, but can we really interpret that as a condition on them?

• You can actually learn the 'best' hyperparameter from the data before you predict, i.e. you maximize a likelihood function w.r.t. the data (see the slide "GP learning the kernel" in mlss2011.comp.nus.edu.sg/uploads/Site/lect1gp.pdf). However, you may overfit when doing so and the author claims that one can use Bayesian methods in order to cope for that but I have never done that before... You could also add some kind of regularization (which, however, introduces a new hyperparameter, namely the weight for the punishment factor in the likelihood) – Fabian Werner Dec 5 '18 at 7:55