1
$\begingroup$

in deep learning and machine learning, when we increse the number of batch size then we should increse the learning rate and decrese the max-steps(iterations). Also, I know that the number of epochs based on the dataset (eg. contains one objects or various of objects).

If we suppose that the number of dataset equal to 1453 samples and the batch size is equal to 32. what are the best initial learning rate, max-steps, and decay rate as starting? and there is an general equation for this?????

$\endgroup$
  • $\begingroup$ There is no general equation and cannot be, it depends of course on the optimization problem, though there can be heuristics for specific NN and problem types. As for reasonable parameter values, try to look at existing code on GitHub and other places (code using Keras is easier to read). $\endgroup$ – John Donn Jan 22 '18 at 11:42
1
$\begingroup$

As John Donn said, there is no general equation and in most cases it is a trial and error process. Often it strictly depends by your data, so try to start with just some standard settings which has already been tested in some similar work.

Since that the parameters that you mentioned depend from the optimization algorithm you are using, you can try some variant of the stochastic gradient descent, like AdaDelta which probably won't give you the best results in terms of accuracy, but at least there you don't need to set all of those parameters.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.