I've currently implemented a neural network and I want to know which works better, the sequential learning or the batch learning. As far as I understand, the batch learning is more efficient than the sequential learning, but according to the tests I've made, there is barely any difference between the accuracy of the two learning styles. Worth to notice is that I have a binary classification problem. My question is therefore, is there anything I can do to really showcase the difference between them, is one better for smaller type of training, more complex decision boundaries etc? Thanks in advance!
$\begingroup$
$\endgroup$
5
-
$\begingroup$ Batch method is good and gives some accurate results when compared to training set to the testing set. $\endgroup$– Narayana MurthyCommented Jan 31, 2019 at 12:47
-
$\begingroup$ When you say "sequential learning" do you mean submitting your entire dataset as one single large batch, rather than splitting into smaller batches? $\endgroup$– BigBadMeCommented Jan 31, 2019 at 12:59
-
$\begingroup$ I mean that if I have a large data-set with 50 patterns, I submit 1 pattern at a time $\endgroup$– Konrad SCommented Jan 31, 2019 at 13:05
-
$\begingroup$ I'm more interested in a comparison, as far as i understand batch is more efficient in most cases cause of the efficient matrix calculations, but when would you in that case use sequential? $\endgroup$– Konrad SCommented Jan 31, 2019 at 13:06
-
$\begingroup$ I don't think it's ever recommended to do truly sequential, i,e, batch size = 1, because it's just so stochastic and noisy; you'd be doing weight updates based on just one example of data. $\endgroup$– BigBadMeCommented Jan 31, 2019 at 14:24
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
Andrew Ng gives a good explanation here on how batch sizes affect the training of a network:
https://www.youtube.com/watch?v=2GGKR-vOlgo
Essentially, if you set your mini batch size to 1 (sequential as you refer to it) then it can cause the learning to behave very stochastically (noisy), and if you set it too small (or to 1) then it's unlikely to converge on the minima.
$\begingroup$
$\endgroup$
Batch learning is preferred to Sequential learning as its more time consuming and your R2 might prune to over-saturation. Try both if you are doing it for the first time and share the feedback.
