Timeline for What is the trade-off between batch size and number of iterations to train a neural network?
Current License: CC BY-SA 3.0
3 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 16, 2020 at 17:08 | comment | added | jlh | More memory is not necessarily required for larger batch sizes. One common method is to accumulate the gradients over several normal sized batches and then perform one single weight update using the average/sum of the gradients. This results in virtually using a larger batch size. Example: stackoverflow.com/questions/46772685/… | |
| May 24, 2018 at 21:23 | comment | added | P-Gn | A larger memory requirement seems a bad trade-off for simply avoiding to decrease a value. Also IMHO having the memory footprint growing during training makes for a less, not more, scalable algorithm. | |
| Mar 7, 2018 at 2:51 | history | answered | David Parks | CC BY-SA 3.0 |