I am working on tuning a machine learning model and want to perform a grid search / hyperparameter tuning on my model to find the best hyperparameters.

The literature I have found it pretty good with explaining the different methods of tuning but now about how they select or justify the ranges they select to tune the parameters.

For example, I want to turn the batch size and learning rate parameters in my model. Right now, the batch size is 8 and the learning rate is 0.0015 (selected arbitrarily). If I were to tune these parameters, how would I know the minimum / maximum values to test.

In the end I need to tune about 7 parameters, so I am looking for some type of rule-of-thumb or justification from literature, or if anyone can offer general advice in this topic.

  • $\begingroup$ learning rate: bt 1e-6 and 1e-2. batch size: between 16 and as much as will fit on your gpu. $\endgroup$ – shimao Nov 25 '19 at 22:00
  • $\begingroup$ what's a reasonable range just comes with experience -- you gain an intuition for what is likely to work and what will not $\endgroup$ – shimao Nov 25 '19 at 22:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.