0
$\begingroup$

I'm doing stochastic gradient descent on a non-convex optimization problem. Gradient corresponds to an intractable expectation which I approximate via Monte Carlo averaging. I'm trying to infer the values of a vector (corresponding to my parameters). One of the dimensions of the vector has much higher variance compared to the rest. I'm not converging to a good local minimum. Are there any efficient variance reducing gradient descent methods I could use instead? I'm aware of SVRG however not familiar at the moment with the state of the art in this area.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.