0
$\begingroup$

I've currently implemented a neural network and I want to know which works better, the sequential learning or the batch learning. As far as I understand, the batch learning is more efficient than the sequential learning, but according to the tests I've made, there is barely any difference between the accuracy of the two learning styles. Worth to notice is that I have a binary classification problem. My question is therefore, is there anything I can do to really showcase the difference between them, is one better for smaller type of training, more complex decision boundaries etc? Thanks in advance!

$\endgroup$
5
  • $\begingroup$ Batch method is good and gives some accurate results when compared to training set to the testing set. $\endgroup$ Jan 31, 2019 at 12:47
  • $\begingroup$ When you say "sequential learning" do you mean submitting your entire dataset as one single large batch, rather than splitting into smaller batches? $\endgroup$
    – BigBadMe
    Jan 31, 2019 at 12:59
  • $\begingroup$ I mean that if I have a large data-set with 50 patterns, I submit 1 pattern at a time $\endgroup$
    – Konrad S
    Jan 31, 2019 at 13:05
  • $\begingroup$ I'm more interested in a comparison, as far as i understand batch is more efficient in most cases cause of the efficient matrix calculations, but when would you in that case use sequential? $\endgroup$
    – Konrad S
    Jan 31, 2019 at 13:06
  • $\begingroup$ I don't think it's ever recommended to do truly sequential, i,e, batch size = 1, because it's just so stochastic and noisy; you'd be doing weight updates based on just one example of data. $\endgroup$
    – BigBadMe
    Jan 31, 2019 at 14:24

2 Answers 2

0
$\begingroup$

Andrew Ng gives a good explanation here on how batch sizes affect the training of a network:

https://www.youtube.com/watch?v=2GGKR-vOlgo

Essentially, if you set your mini batch size to 1 (sequential as you refer to it) then it can cause the learning to behave very stochastically (noisy), and if you set it too small (or to 1) then it's unlikely to converge on the minima.

$\endgroup$
2
  • $\begingroup$ Thank you for this, the video was actually really helpful, got an idea of where to go from here $\endgroup$
    – Konrad S
    Jan 31, 2019 at 14:15
  • $\begingroup$ No problem. All of Andrew Ng's video are certainly worth a watch. $\endgroup$
    – BigBadMe
    Jan 31, 2019 at 14:16
0
$\begingroup$

Batch learning is preferred to Sequential learning as its more time consuming and your R2 might prune to over-saturation. Try both if you are doing it for the first time and share the feedback.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.