Does anyone know how to construct a confidence interval for predicting a new test value given a trained Relevance Vector Machine (rvm) and/or Gaussian Process Regression (gausspr) using the kernlab R package?

More specifically, how do I get:

  1. The standard error/deviation (variance) of a new test point;

  2. The parameters estimates of posterior distribution of the parameters?

I would appreciate if anyone could point to a document that discuss how to obtain/calculate the above, from the output of function call (rvm or gausspr).


Ok, here's the scoop: first off, have a look at my class in Berkeley. The slides for GP regression should be up shortly at http://alex.smola.org/teaching/berkeley2012 where I discuss this in more detail.

For a GP regression you can integrate out the latent GP and since the observation model and the underlying GP are both Gaussian, you get a Gaussian again. In math this means that

$$y = t + \epsilon.$$

Here $t$ is drawn from a GP and $\epsilon$ are independent Gaussian random variables. Once you have the GP representation for $y$, you can simply condition on the observed terms to get the residual variance for the rest. This yields something of the form

$$E[y'|y] = \mu' + K_{x'x} K_{xx}^{-1} (y - \mu)$$

and the variance is

$$K_{x'x'} - K_{x'x} K_{xx}^{-1} K_{xx'}.$$

Obviously here we added the noise for $\epsilon$ to the main diagonal. For the relevance vector machine this isn't quite so trivial since it assumes a rather different noise model. So you cannot integrate out the latent variables explicitly. Have a look at my book 'Learning with Kernels' or alternatively Mike Tipping's original paper for the details.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.