Is it possible to perform an approximated fully Bayesian (1) selection of hyper-parameters (e.g. covariance scale) with the GPML code, instead of maximizing the marginal likelihood (2) ? I think using MCMC methods to solve the integrals involving hyper-parameters prior should lead to better results when dealing with overfitting. Up to my knowledge, the GPML framework doesn't include these computations, but perhaps there are goods other third party codes.

(1) Sec. 5.2, Ch. 5 in Gaussian Process for Machine Learning, Rasmussen & Williams, 2006

(2) Section "Regression" in the GPML documentation

  • $\begingroup$ Have you heard of INLA? may be what you're after. $\endgroup$ – probabilityislogic Sep 11 '13 at 10:12
  • $\begingroup$ This is not adding to your question, but have you managed to find useful work in this area of putting priors on length scales? Absolutely hate the idea of just numerically optimising the length scales of a GP $\endgroup$ – sachinruk Nov 15 '13 at 4:20
  • 1
    $\begingroup$ (+1) Great question. This is no MCMC, but there are third-party packages that allow partial marginalization of hyperparameters with GPML via Laplace approximation, if you're interested. See this question and related answers. $\endgroup$ – lacerbi Apr 1 '16 at 20:38

There is another package for machine learning using Gaussian processes called GPstuff which has it all in my opinion. You can use MCMC, integration on a grid, etc. to marginalise out your hyperparameters.

NB In the documentation they call hyperparameters merely parameters.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.