I am currently using the GPML toolbox to perform regression.

Generally, after learning the hyperparameters we can extract the posterior mean and variance by using the function in the toolbox as

[m s2] = gp(hyp2, @infExact, [], covfunc, likfunc, x, y, z);

Here, we can say that m is the Posterior mean and s2 is the posterior variance.
But the code only gives out the self-variance or variance at the target points and not the full covariance matrix.

$s2 = diagonal(k(x_{target},x_{target}'))$
A vector with only the diagonal terms.

How can I get the full covariance matrix between all the target points i.e. $k(x_{target},x_{target}')$?

I tried writing something like:

kTT = feval(covFunc{:}, hyp2.cov, xTarget);
kTI = feval(covFunc{:}, hyp2.cov, xTarget,x);
kII = feval(covFunc{:}, hyp2.cov, x);
sigma = exp(2*hypProd.lik)*eye(max(size(x))); 

% Posterior covariance matrix
kPost = kTT - kTI/(kII+sigma)*kTI';

But the diagonal variance values in the kPost matrix don't match those with the s2 matrix. What am I doing wrong?

  • $\begingroup$ Can you clarify what you're question is about? What are your data? What software are you using? Etc. $\endgroup$ – gung - Reinstate Monica Jan 16 '15 at 15:56
  • $\begingroup$ @gung I have edited the question, is it still confusing? $\endgroup$ – Ankit Chiplunkar Jan 19 '15 at 8:52

As you already answered yourself, the m and s2 output variables are just the means and variances of prediction at the given points z.

However, I'd like to add that you can also get a posterior structure by calling the function as

[predmeans, predvars, latentmeans, latentvars, [], poststr] = gp(hyp, inffun, meanfun, covfun, likfun, xtrain, ytrain, xtest)

In this case, poststr is going to be a structure consisting of three fields: alpha, sW and L which are supposed to be used to construct the posterior means and variances using the formula given in the GPML toolbox for Matlab manual, page 4.

Also of note: this posterior structure can also be returned when training (instead of predicting, as in your example as well as the one I wrote out above), using a function call like

[negloglik, derivnegloglik, poststr] = gp(hyp, inffun, meanfun, covfun, likfun, xtrain, ytrain)

I finally got the problem, just writing the answer here so that it might be helpful to others.

After enough digging around I figured that:
s2 is the variance of $y_*$
Whereas, kPost is the covariance of $f_{*}$

If we write the posterior of "f" $f_{*} | x_{*}, y, x = GP(K_{*}(K+\sigma_{n}^{2}I)^{-1}y, K_{**}-K_{*}^{T}(K+\sigma_{n}^{2}I)^{-1}K_{*})$
This is equivalent to the kPost that I calculate in the question above.

But the GPML gives as an output $y_{*} | x_{*}, y, x = GP(K_{*}(K+\sigma_{n}^{2}I)^{-1}y, K_{**}-K_{*}^{T}(K+\sigma_{n}^{2}I)^{-1}K_{*}+\sigma_{n}^{2}I)$

The $y_*$ has an extra term in the covariance matrix which is the noise in the measurements.

Hence, $s2 = diagonal(kPost+\sigma_{n}^{2}I)$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.