Timeline for Can I checking the correct implementation for gradient descent algorithm by looking at if the loss is monotonically decreasing?
Current License: CC BY-SA 3.0
11 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Aug 24, 2016 at 18:54 | answer | added | A.D | timeline score: 2 | |
| Aug 24, 2016 at 18:20 | answer | added | horaceT | timeline score: 3 | |
| Aug 24, 2016 at 18:05 | comment | added | horaceT | BTW, not an expert myself, but theano and friends (python world) spare you the headache of coming up with theoretical gradient. They do symbolic differentiation so you just specify the objective function and voila comes the gradient! | |
| Aug 24, 2016 at 17:57 | comment | added | horaceT | A while back I was coding a flavor of deep neural net and I found the 'numDeriv' package quite useful. You feed the 'grad' function with your loss and it spits out the derivative calculated at a point. | |
| Aug 24, 2016 at 17:45 | comment | added | Haitao Du | @horaceT, i agree, however, in my case, even numerical gradient is a little bit hard to calculate. | |
| Aug 24, 2016 at 17:40 | comment | added | horaceT | The best method of validating any gradient descent algorithm is validating the gradient. The usual approach is to get a numeric gradient and check against your calculated version. | |
| Aug 24, 2016 at 15:12 | vote | accept | Haitao Du | ||
| Aug 24, 2016 at 14:39 | answer | added | Mark L. Stone | timeline score: 7 | |
| Aug 24, 2016 at 13:56 | history | edited | Haitao Du | CC BY-SA 3.0 |
added 1 character in body; edited title
|
| Aug 24, 2016 at 13:00 | history | edited | Haitao Du | CC BY-SA 3.0 |
added 18 characters in body; edited tags; edited title
|
| Aug 24, 2016 at 5:57 | history | asked | Haitao Du | CC BY-SA 3.0 |