In the deep learning process, especially SAR-ATR (e.g., generic object detection), is there any relation between the training time (speed) and the training batch size?
Is there a paper related to this question?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ Even if you read data, without any additional operations, then with larger files it takes longer then with smaller files. $\endgroup$– TimCommented Jan 28, 2019 at 14:47
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
There's no exact formula, but usually there's some kind of a optimal batch size. Batch size 1 or batch size equal to entire training sample size usually run slower than something between these extreme, e.g. 100. You'll have to find what's the optimal size for your problem and ML software/hardware setup.
$\begingroup$
$\endgroup$
If you use GPU you can usually achieve better performance (speed-up) by increasing batch size, this because of matrix multiplication and using multi-threading in better way. Bigger batch leads optimization step to be more "directed by data" (less random walking), however it could in rare situation leads to stack in a local minimum. You should be aware that increasing batch size could also leads to need of no epochs increasing.
Some authorities recommend to use the biggest batch you can.
