Stack Exchange network consists of 183 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.
This answer is misleading. It says that SGD implies batch_size=1, which might be true in some old textbooks, but is just plain wrong in modern practice. Everybody uses minibatches with SGD because GPUs. I agree that full batch gradient descent is smoother, but in modern practice most interesting datasets are too large for for full batch GD.
