Timeline for How to choose between SGD with Nesterov momentum and Adam?
Current License: CC BY-SA 3.0
5 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 17, 2018 at 21:59 | comment | added | Shuklaswag | Why not use both? web.stanford.edu/~tdozat/files/TDozat-CS229-Paper.pdf | |
| Apr 24, 2018 at 17:56 | answer | added | Austin Shin | timeline score: 15 | |
| Nov 24, 2017 at 10:01 | comment | added | denis | downhill does AdaM etc. with or without Nesterov momentum (wraps theano). Could you run a problem similar to yours on that -- try both ? | |
| Jun 2, 2017 at 13:18 | comment | added | Sycorax♦ | this is a nice article: sebastianruder.com/optimizing-gradient-descent but in general, which is best depends on the network architecture and how much time you want to spend tuning the optimizer | |
| Jun 2, 2017 at 13:07 | history | asked | Clément F | CC BY-SA 3.0 |