2
$\begingroup$

I'm working through Rasmussen's Gaussian Processes book, and I have a question about the possibility of optimizing additional basis function hyperparameters (in section 2.7 http://www.gaussianprocess.org/gpml/chapters/RW2.pdf). The text explains that the hyperparameters can be varied to maximize the log marginal likelihood, which is given in eq. 2.44 and 2.45. The derivative for the general case is given in eq. 5.9.

I've looked in the gpml code that goes with the book and relevant literature, but I haven't been able to find the derivative wrt the hyperparameters including the basis function hyperparameters.

In other words, for anyone familiar with Gaussian Process Regression, what is the derivative of eq. 2.44 wrt the hyperprior? (again, GPR Book, Ch. 2).

Improve this question
$\endgroup$
1
  • $\begingroup$ Be aware that if you have too many hyper-parameters it can lead to over-fitting the marginal likelihood, resulting in a model that gives worse predictions. For instance marginal likelihood maximisation with an Automatic Relevance Determination (ARD) kernel is often out-performed by a model with a simple RBF kernel, see the paper I wrote on this with Mrs Marsupial jmlr.org/papers/volume11/cawley10a/cawley10a.pdf Marginalising over the hyper-parameters, rather than optimising, may be a better approach in such cases. $\endgroup$
    – Dikran Marsupial
    Aug 20, 2021 at 6:46

1 Answer 1

Reset to default
1
$\begingroup$

Check out my post here. It answers your question exactly. If you wish to do it for more exotic kernels either check out a text on kernel methods which might help you or try performing the partial derivatives yourself.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.