I've read this regarding the difference between epoch and mini-batch.

To clarify:

  • With an epoch value of 1000 and batch size of 50, does that mean that the model will use each data point exactly 1000 times in such an (random) order where at each iteration only 50 data points are used for optimization? (meaning a total of 50*1000 calculations?)
  • Is every data point used exactly 1000 times?

Yes, each iteration uses mini-batch number of points (here 50) and there will be 20 iterations in an epoch. An epoch ends when we go over all the training samples exactly once. This means every training point is used exactly 1000 times.

  • $\begingroup$ I see, so the number of iterations is epoch / batch-size and the number of total calculations is epoch * number-of-points (as each point is used to calculate the rate of change of error epoch amount of times?) $\endgroup$ – alexandrosangeli Feb 2 at 17:35
  • $\begingroup$ if by calculation you mean a training sample going through the alg, yes that’s correct $\endgroup$ – gunes Feb 2 at 17:37
  • $\begingroup$ Yes that's what I mean. Thank you. $\endgroup$ – alexandrosangeli Feb 2 at 17:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.