I was doing CS231n assignments and found a very interesting implementation of mini-batch gradient descent for SVM image classifier assignment.
for each epoch: sampling some random 'batch_size' number of examples from the training data grad = finding gradient over these sampled examples changing weights using the calculated gradient
This is pretty weird comparing to the mini-batch gradient descent which is used in neural networks for instance. It's something like :
for each epoch: batches = create batches of training data using whatever the batch_size is for each batch in batches: grad = calculate gradient over examples in this 'batch' of training data changing weights using this above calculated gradient
This one makes much more sense to me as compared to the above one because it uses all the examples for making changes in that epoch, unlike the CS231n's code which uses just some random 'batch_size' number of examples in each epoch.
Can someone explain this to me?