Timeline for Spikes in loss function of deep neural network training [duplicate]
Current License: CC BY-SA 4.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Oct 26, 2018 at 14:56 | history | closed |
Sycorax♦ kjetil b halvorsen♦ mdewey Ferdi whuber♦ |
Duplicate of Explanation of Spikes in training loss vs. iterations with Adam Optimizer | |
| Oct 25, 2018 at 0:30 | review | Close votes | |||
| Oct 26, 2018 at 14:56 | |||||
| Oct 24, 2018 at 13:42 | comment | added | Sycorax♦ | Possibly related: stats.stackexchange.com/questions/169216/… although the fact that this disappears when using SGD instead of Adam suggests that the difference is related to the optimizer. | |
| Oct 24, 2018 at 13:37 | history | edited | Ferdi | CC BY-SA 4.0 |
deleted 7 characters in body
|
| Apr 15, 2018 at 0:45 | history | tweeted | twitter.com/StackStats/status/985318204821131264 | ||
| Sep 13, 2017 at 7:20 | comment | added | user38320 | @Aaron, yes the problem is similar, but in the other post the loss diverges, while in mine the trend is not changed by the spikes: their effects are only local. | |
| Sep 12, 2017 at 22:52 | comment | added | Aaron | Seems like it could be the same problem as in the other post that you linked. | |
| Sep 12, 2017 at 11:09 | history | asked | user38320 | CC BY-SA 3.0 |