Timeline for Why is a 2nd order derivative optimization better for no hidden layer neural networks?
Current License: CC BY-SA 4.0
4 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Oct 24, 2021 at 18:43 | vote | accept | user8714896 | ||
| Oct 24, 2021 at 14:58 | comment | added | shimao | ah yes, that is a good catch | |
| Oct 24, 2021 at 14:55 | comment | added | Dikran Marsupial | I suspect it is the "stochastic" in SGD that is preventing it from converging to the exact minima as the updates are noisy as they depend on the batch used in each iteration. I don't think it is a clean test of first-order versus second order optimisation. | |
| Oct 24, 2021 at 14:17 | history | answered | shimao | CC BY-SA 4.0 |