# Gradient ascent to maximise log likelihood

I'm working on an online method to adapt the parameters $\mu, \Sigma$ of a Gaussian distribution. Do to so i perform a gradient descent on the log likelihood $L$. With the help of the matrix cookbook i've found that the partial derivatives $\nabla_\Sigma L$ and $\nabla_\mu L$ are given by

\begin{align} \nabla_\Sigma L = \frac{ \partial {\bf L} }{ \partial {\boldsymbol \Sigma}} &= -\frac{1}{2} \left( {\boldsymbol \Sigma}^{-1} - {\boldsymbol \Sigma}^{-1} \left( {\boldsymbol y} - {\boldsymbol \mu} \right) \left( {\boldsymbol y} - {\boldsymbol \mu} \right)' {\boldsymbol \Sigma}^{-1} \right) \end{align} and \begin{align} \nabla_\mu L = \frac{ \partial {\bf L} }{ \partial {\boldsymbol \mu}} &= {\boldsymbol \Sigma}^{-1} \left( {\boldsymbol y} - {\boldsymbol \mu} \right) \end{align} where $\boldsymbol{y}$ are the training samples and $\boldsymbol{L}$ the log likelihood of the multivariate gaussian distribution given by $\mu$ and $\Sigma$. I'm setting a learning rate $\alpha$ and proceed in the following way:

1. Sample an $y$ from unknown $p_\theta(y)$.
2. Compute gradients $\nabla_\Sigma L$ and $\nabla_\mu L$
3. Update: $\mu = \mu + \nabla_\mu L \alpha$, and $\Sigma = \Sigma + \nabla_\Sigma L \alpha$

I've set $\alpha = 0.002$ and $p_\theta(y) = \mathcal{N}(y | 0, 1)$, but it is not convering at all. What am i missing?

P.S.: I know there are different ways, but i'm trying to this using a gradient ascent methood.

• Could you please explain your second equation in pt number 3? – Sid Apr 9 '18 at 22:08
• Sorry, that was a mistake. I fixed it. – hh32 Apr 10 '18 at 5:07
• Just to make sure the error is nothing overly simple: For the univariate case we have $N(y|0,1) = ce^{-(y-\mu)^2/2\Sigma}$ so that $-\log \text{Likelihood} = \text{const} + -(-(y-\mu)^2/2\Sigma) = \text{const} + (y-\mu)^2/2\Sigma$ so that the derivative is $\partial_\mu -\log L = \Sigma^{-1} (y-\mu)$. You want to maximize the likelihood, i.e. maximize $\log L$ i.e. minimize $-\log L$. The gradient points into the direction of the steepest increase in function values, i.e. shouldn't you subtract the gradient in order to minimize? – Fabian Werner Apr 10 '18 at 7:10
• @FabianWerner what you wrote is correct, but I'm taking the derivative of $\log L$, not $-\log L$ so i add the gradient. It should be same. – hh32 Apr 10 '18 at 9:04
• @hh32: that is exactly my point: the partial of -log L (w.r.t. $\mu$) and the one of log L coincide. One of us must have made a mistake... I think you are taking the gradient of -log L, hence you need to minimize, hence subtract... or was it me who committed the error? – Fabian Werner Apr 10 '18 at 11:58

## 1 Answer

One possibility is that the sample covariance matrix is becoming near singular somewhere along the way, and your iterations involve inverting this matrix

The eigenvalues of your sample covariance matrix will follow a Marcenko-Pastur distribution (https://en.wikipedia.org/wiki/Marchenko%E2%80%93Pastur_distribution), and from this distribution you will see that it is not inconceivable that you get high condition numbers (the condition number is the ratio of the largest to the smallest eigenvalue of your covariance matrix)