Relationship between variance of gradient and SGD convergence

Question

I've found things such as the Robbins-Monroe conditions for the learning rate, as well as a proof from Robbins, Siegmund, 1971 which gives convergence to a local minima provided that the expectation and variance of the gradient are both bounded.

I'm looking for a reference that talks about the relationship between the variance of the gradient and convergence of SGD. Specifically, I know that lower variance is good, but given that the gradient is a vector, how is "lower" defined, and what are the guaranteed benefits of reducing the variance?

Answer 1

Only in response to:

I'm looking for a reference that talks about the relationship between the variance of the gradient and convergence of SGD.

Whilst the Robbins-Monro stochastic approximation paper is the seminal reference, its scope is broader than that of stochastic gradient descent. There are more contemporary papers that cover stochastic approximation specifically in context of stochastic gradient descent.

For some theoretical results concerning the gradient variance and convergence, an appropriate starting point might be:

Bottou, L., Curtis, F., Nocedal, J. (2018). Optimization Methods for Large-Scale Machine Learning. SIAM Review Vol 60 Issue number 2 pp 223-311. DOI:10.1137/16M1080173

The bibliography of that paper and other papers by Leon Bottou and associates may also yield fruit.