2
$\begingroup$

What impact does mini-batch size have on Adam Optimizer? Is there a recommended mini-batch size when training a (covolutional) neural network with Adam Optimizer?

From what I understood (I might be wrong), for small mini-batch sizes the results tend to be noisy, but the results are also innacurrate for big batches (let's say the whole training set in one pass).

$\endgroup$
3
$\begingroup$

I would just leave this as a comment, but I don't have enough reputation.

There's an excellent discussion of the trade offs of large and small batch sizes here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.