0
$\begingroup$

I understand that stochastic gradient descent has a batch size of 1, but while reading inception v2 paper, I found this text in training methodology "We have trained our networks with stochastic gradient utilizing the TensorFlow [1] distributed machine learning system using 50 replicas running each on a NVidia Kepler GPU with batch size 32 for 100 epochs." can anyone help me out with this?

$\endgroup$
3
$\begingroup$

There are two common meanings of “stochastic gradient decent” in the literature. One holds that SGD uses a single example per iteration. The second holds that using “small” batches of 1 or more samples is SGD.

There’s no particular reason that either definition is “more correct” than the other; however, there are often large practical and experimental differences for using more than one example per iteration.

| cite | improve this answer | |
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.