I read from various sources that Gaussian process (GP) is a promising way to do hyperparameter search/optimization for ML/DL model. However, GP itself also has hyperparameters such as length-scale, the signal variance and the noise variance. So how is GP a valid way of doing hyperparameter search if itself also needs hyperparameter search (same problem still persists)?
2
-
$\begingroup$ The hyperparameters of GP can be found in a few seconds when the sample size isn't too big (say less than a hundred or so). If fitting your ML model at one hyperparameter setting takes say 1 hour, the GP problem is trivial in comparison to the hyperparameter problem. $\endgroup$– jckenCommented Feb 9, 2021 at 7:19
-
$\begingroup$ Thank you for your reply. Could you also recommend some reference materials on how to do hyperparameter tuning for GP? $\endgroup$– SamCommented Feb 10, 2021 at 15:17
Add a comment
|