Timeline for Gradient ascent to maximise log likelihood
Current License: CC BY-SA 3.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Apr 12, 2018 at 11:23 | vote | accept | hh32 | ||
| Apr 11, 2018 at 4:44 | answer | added | Sid | timeline score: 1 | |
| Apr 10, 2018 at 11:58 | comment | added | Fabian Werner | @hh32: that is exactly my point: the partial of -log L (w.r.t. $\mu$) and the one of log L coincide. One of us must have made a mistake... I think you are taking the gradient of -log L, hence you need to minimize, hence subtract... or was it me who committed the error? | |
| Apr 10, 2018 at 9:04 | comment | added | hh32 | @FabianWerner what you wrote is correct, but I'm taking the derivative of $\log L$, not $-\log L$ so i add the gradient. It should be same. | |
| Apr 10, 2018 at 7:10 | comment | added | Fabian Werner | Just to make sure the error is nothing overly simple: For the univariate case we have $N(y|0,1) = ce^{-(y-\mu)^2/2\Sigma}$ so that $-\log \text{Likelihood} = \text{const} + -(-(y-\mu)^2/2\Sigma) = \text{const} + (y-\mu)^2/2\Sigma$ so that the derivative is $\partial_\mu -\log L = \Sigma^{-1} (y-\mu)$. You want to maximize the likelihood, i.e. maximize $\log L$ i.e. minimize $-\log L$. The gradient points into the direction of the steepest increase in function values, i.e. shouldn't you subtract the gradient in order to minimize? | |
| Apr 10, 2018 at 5:07 | comment | added | hh32 | Sorry, that was a mistake. I fixed it. | |
| Apr 10, 2018 at 5:07 | history | edited | hh32 | CC BY-SA 3.0 |
mistake
|
| Apr 9, 2018 at 22:08 | comment | added | Sid | Could you please explain your second equation in pt number 3? | |
| Apr 9, 2018 at 19:50 | history | asked | hh32 | CC BY-SA 3.0 |