I mainly read this blog. And this blog sites this paper for the statement that Adagrad improved the robustness of SGD.

I have tried to check the original paper or other articles which explains why Adagrad improves the robustness compared with SGD.However, I couldn't find any reasonable explanation even in the cited paper.

I think Adagrad has greate advantages including per-parameters learning rate and needlessness of setting the learning rate at initial stage.

Could anyone explain why someone says Adagrad improved the robustness of SGD?

Or in the first place, is it true?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.