# Hyperparameters Optimisation in Gaussian process for regression

I am trying to perform Gaussian Process for regression. I chose the SE Kernel : $K(x_i,x_j) = \exp(-\frac{||x_i-x_j||^2}{l}) + \sigma_n\delta_{i,j}$. I begin by maximize the log-likelihood with respect to the latent variables, with conservative values for hyperparameters.

Then I maximize the log-likelihood with respect to the hyperparameters and the results seems weird : the parameter $\sigma_n$ does not stop increasing. I don't get why the log-likelihood grows always with $\sigma_n$ while the model obviously goes wrong.

I tried also to maximize the log-likelihood only with respect to the length-scale parameter $l$. In this case, the log-likelihood is the highest when $l$ equal the value used to perform the first optimization with respect to the hyperparameters.

I both perform the optimization of the log-likelihood with respect to the latent variables by maximizing

$-\frac{dn}{2} - \frac{d}{2}\ln(|K|) - \frac{1}{2}\text{tr}(K^{-1}YY^T)$

where $Y$ is the $n \times d$ matrix of observed data in the function-space. Is there something I poorly undesrtood ?

$-\frac{1}{2}y^tK^{-1}y - \frac{1}{2}log(|K|) - \frac{d}{2}log(2\pi)$
• Thank you ! Indeed the log-likelihood as i wrote in my thread was wrong. In my case i have $n$ observed points in a $d$-dimensionnal space, that's why i have $-\frac{1}{2}\text{tr}(K^{-1} Y Y^T)$ instead (cf jmlr.org/papers/volume6/lawrence05a/lawrence05a.pdf eq6) About the local minima : do you suggest me to optimize the log-likelihood with respect to the hyperparameters using another set of latent variables than those found by the first maximization ? Last question : how do one put a prior on hyperparameters ? i read rasmussen's book but i am strunggling to understand it by myself. – Irminsul Sep 25 '15 at 11:39