In the RMSProp algorithm (And similar algorithms) that are used in Machine Learning in the subject of Adaptive Learning Rates, the squares of the gradients are used in the algorithm step.

Is there motivation for why the squares of the gradients are used?



Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.