2
$\begingroup$

While studying hyperparameter tuning in Machine Learning, I have come to read Bayesian Optimization for Hyperparameter Tuning and Using validation set when training the model but it is kind of ambiguous to me to tell the usage of Bayesian Optimization and validation set when training the model. Using validation data set, I can measure the performance of the model adjusting the hyperparameters but I don't know when to use Bayesian Optimization for hyperparameter tuning. Hope to hear some explanations about it.

$\endgroup$

1 Answer 1

2
$\begingroup$

Bayesian optimization (BO) is a recipe to tell how you should explore the hyper-parameter (HP) space. So, you'll be using your validation set as before, but exploring the space in the direction that BO suggests. This is typically useful when you have a lot of HPs to tune and it's computationally expensive (or impossible) to try them all. Grid search explores the HP space in an ordered manner; random search just randomly picks some HP values but BO selects the next HPs to try based on how much information it'd gain from trying that HPs with your model. It's much more economic for a large and high dimensional HP space.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.