Timeline for Why does momentum escape from a saddle point in this famous image?
Current License: CC BY-SA 3.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jun 1, 2022 at 20:57 | comment | added | Rafael Toledo | Fixing @Sam's comment, SGD is about parameters updating!, it's an optimizer! The backpropagation algorithm is what computes the gradients. | |
| Oct 20, 2017 at 15:22 | answer | added | Kazuya Tomita | timeline score: 2 | |
| Oct 20, 2017 at 13:32 | comment | added | Sam | Yeah, I think the Gif is strange | |
| Oct 20, 2017 at 2:13 | comment | added | Aaron | The momentum doesn't escape. It's the green line and it stays in the saddle point until the end. | |
| Oct 20, 2017 at 0:18 | history | edited | Kazuya Tomita | CC BY-SA 3.0 |
added 24 characters in body
|
| Oct 20, 2017 at 0:17 | comment | added | Kazuya Tomita | I totally agree with you, so do you think the animated gif is strange, too? | |
| Oct 19, 2017 at 14:42 | comment | added | Sam | Why is SGD (Stochastic Gradient Descent) listed with these other things? SGD is about how to calculate the Gradient, not about how to update it. I don't think SGD's should get stuck in a saddle, because any subset that you choose to calculate the stochastic gradient will have a slightly different error plane, so it should never stop updating (unless you decrease the learning rate to 0) | |
| Oct 19, 2017 at 14:13 | history | asked | Kazuya Tomita | CC BY-SA 3.0 |