I saw a method for dimensionality reduction for the squared exponential covariance function (not ARD) whereby one uses a $G\times D$ projection matrix $P$ ($G < D$, $D$ = dimension of the inputs) such that the squared distances are calculated in a lower dimensional space. E.g. the kernel function $$K(x, x') = c \times\exp\left(-\frac{1}{2}P(x-x')^TP(x-x')\right)$$

I'm trying to use this in my implementation of Gaussian process regression, but I'm not sure how to get the gradient with respect to $P$ for optimization. I need the derivative of the marginal likelihood w.r.t. $P$ to maximize (minimizing the negative log marginal likelihood in actuality). Has anyone seen this method before or have any suggestions?

  • $\begingroup$ can you give a reference to the paper? $\endgroup$ – Dikran Marsupial Jun 14 '12 at 8:52
  • $\begingroup$ Yep no problem. I'm looking at "Variable noise and dimensionality reduction for sparse Gaussian processes", Snelson and Ghahramani. You can see what I've been reading lately. link here: gatsby.ucl.ac.uk/~snelson/snelson_uai.pdf $\endgroup$ – tomas Jun 14 '12 at 12:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.