3
$\begingroup$

I implemented these two methods in a deep learning project where I am using theano. I understand the mathematical difference between these two methods, and my conceptual understanding is that nesterov is an improvement over momentum.

My question is: are there practical situations where momentum descent would be preferred over nesterov? My experience is that nesterov is always better. What would be a situation in which I would use momentum?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.