Following the Rasmussen&Williams Gpml Machine Learning book i'm trying to implement my gaussian process in matlab, avoiding to use other existing toolbox or complex pre-assembled functions, but now i'm stuck with the minimization of the negative log marginal likelihood in order to estimate the hyperparameters of my noisy covariance function and i'm not sure of it. In particular i used the Squared Exponential in the form:
Kxx = sf^2*exp(-0.5*(squareform(pdist(x)).^2)/ell^2)+(sn)^2*eye(Ntr,Ntr);
where sf is the signal standard deviation, ell the charactersitic length scale, sn the noise standard deviation and Ntr the length of the training input data x. Is there any difference with expressing it in the form:
The negative log marginal likelihood is:
where ytr is the training noisy targets vector. Comparing it with that one obtained using Rasmussen's Gpml Toolbox the result is the same.
To optimize the parameters i need to evaluate the gradient of the negative log marginal likelihood. The partial derivatives are expressed by Rasmussen as:
and in my code, i.e. deriving for sf, i obtain:
but the result is completely different (and wrong). I don't know where the problem is, maybe is related with logarithm or the matrix derivatives or computationl precision, but i can't see it.
Thank you in advance for helping!!
The problem were the hyperparamters considered, since often GP libraries use the logarithmic notation as explained here: