I read from various sources that Gaussian process (GP) is a promising way to do hyperparameter search/optimization for ML/DL model. However, GP itself also has hyperparameters such as length-scale, the signal variance and the noise variance. So how is GP a valid way of doing hyperparameter search if itself also needs hyperparameter search (same problem still persists)?

  • $\begingroup$ The hyperparameters of GP can be found in a few seconds when the sample size isn't too big (say less than a hundred or so). If fitting your ML model at one hyperparameter setting takes say 1 hour, the GP problem is trivial in comparison to the hyperparameter problem. $\endgroup$
    – jcken
    Feb 9, 2021 at 7:19
  • $\begingroup$ Thank you for your reply. Could you also recommend some reference materials on how to do hyperparameter tuning for GP? $\endgroup$
    – Sam
    Feb 10, 2021 at 15:17


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.