# Posterior covariance from GPML toolbox

I am currently using the GPML toolbox to perform regression.

Generally, after learning the hyperparameters we can extract the posterior mean and variance by using the function in the toolbox as

[m s2] = gp(hyp2, @infExact, [], covfunc, likfunc, x, y, z);


Here, we can say that m is the Posterior mean and s2 is the posterior variance.
But the code only gives out the self-variance or variance at the target points and not the full covariance matrix.

$s2 = diagonal(k(x_{target},x_{target}'))$
A vector with only the diagonal terms.

How can I get the full covariance matrix between all the target points i.e. $k(x_{target},x_{target}')$?

I tried writing something like:

kTT = feval(covFunc{:}, hyp2.cov, xTarget);
kTI = feval(covFunc{:}, hyp2.cov, xTarget,x);
kII = feval(covFunc{:}, hyp2.cov, x);
sigma = exp(2*hypProd.lik)*eye(max(size(x)));

% Posterior covariance matrix
kPost = kTT - kTI/(kII+sigma)*kTI';


But the diagonal variance values in the kPost matrix don't match those with the s2 matrix. What am I doing wrong?

• Can you clarify what you're question is about? What are your data? What software are you using? Etc. – gung - Reinstate Monica Jan 16 '15 at 15:56
• @gung I have edited the question, is it still confusing? – Ankit Chiplunkar Jan 19 '15 at 8:52

As you already answered yourself, the m and s2 output variables are just the means and variances of prediction at the given points z.

However, I'd like to add that you can also get a posterior structure by calling the function as

[predmeans, predvars, latentmeans, latentvars, [], poststr] = gp(hyp, inffun, meanfun, covfun, likfun, xtrain, ytrain, xtest)


In this case, poststr is going to be a structure consisting of three fields: alpha, sW and L which are supposed to be used to construct the posterior means and variances using the formula given in the GPML toolbox for Matlab manual, page 4.

Also of note: this posterior structure can also be returned when training (instead of predicting, as in your example as well as the one I wrote out above), using a function call like

[negloglik, derivnegloglik, poststr] = gp(hyp, inffun, meanfun, covfun, likfun, xtrain, ytrain)


I finally got the problem, just writing the answer here so that it might be helpful to others.

After enough digging around I figured that:
s2 is the variance of $y_*$
Whereas, kPost is the covariance of $f_{*}$

If we write the posterior of "f" $f_{*} | x_{*}, y, x = GP(K_{*}(K+\sigma_{n}^{2}I)^{-1}y, K_{**}-K_{*}^{T}(K+\sigma_{n}^{2}I)^{-1}K_{*})$
This is equivalent to the kPost that I calculate in the question above.

But the GPML gives as an output $y_{*} | x_{*}, y, x = GP(K_{*}(K+\sigma_{n}^{2}I)^{-1}y, K_{**}-K_{*}^{T}(K+\sigma_{n}^{2}I)^{-1}K_{*}+\sigma_{n}^{2}I)$

The $y_*$ has an extra term in the covariance matrix which is the noise in the measurements.

Hence, $s2 = diagonal(kPost+\sigma_{n}^{2}I)$