When using stochastic gradient descent, how do we pick a stopping criteria?
A benefit of stochastic gradient descent is that, since it is stochastic, it can avoid getting stuck in a suboptimal region.
So I don't understand how we can pick a stopping criteria, because surely then we would be limiting the algorithm's ability to "unstuck" itself?
For gradient descent, I would typically use the norm of the gradient as a stopping criteria (so we stop when that norm is small enough).
Would this make sense for SGD too?