I'm working on an online method to adapt the parameters $\mu, \Sigma$ of a Gaussian distribution. Do to so i perform a gradient descent on the log likelihood $L$. With the help of the matrix cookbook i've found that the partial derivatives $\nabla_\Sigma L$ and $\nabla_\mu L$ are given by

\begin{align} \nabla_\Sigma L = \frac{ \partial {\bf L} }{ \partial {\boldsymbol \Sigma}} &= -\frac{1}{2} \left( {\boldsymbol \Sigma}^{-1} - {\boldsymbol \Sigma}^{-1} \left( {\boldsymbol y} - {\boldsymbol \mu} \right) \left( {\boldsymbol y} - {\boldsymbol \mu} \right)' {\boldsymbol \Sigma}^{-1} \right) \end{align} and \begin{align} \nabla_\mu L = \frac{ \partial {\bf L} }{ \partial {\boldsymbol \mu}} &= {\boldsymbol \Sigma}^{-1} \left( {\boldsymbol y} - {\boldsymbol \mu} \right) \end{align} where $\boldsymbol{y}$ are the training samples and $\boldsymbol{L}$ the log likelihood of the multivariate gaussian distribution given by $\mu$ and $\Sigma$. I'm setting a learning rate $\alpha$ and proceed in the following way:

1. Sample an $y$ from unknown $p_\theta(y)$.
2. Compute gradients $\nabla_\Sigma L$ and $\nabla_\mu L$
3. Update: $\mu = \mu + \nabla_\mu L \alpha$, and $\Sigma = \nabla_\Sigma L \alpha$

I've set $\alpha = 0.002$ and $p_\theta(y) = \mathcal{N}(y | 0, 1)$, but it is not convering at all. What am i missing?

P.S.: I know there are different ways, but i'm trying to this using a gradient ascent methood.

I'm working on an online method to adapt the parameters $\mu, \Sigma$ of a Gaussian distribution. Do to so i perform a gradient descent on the log likelihood $L$. With the help of the matrix cookbook i've found that the partial derivatives $\nabla_\Sigma L$ and $\nabla_\mu L$ are given by

\begin{align} \nabla_\Sigma L = \frac{ \partial {\bf L} }{ \partial {\boldsymbol \Sigma}} &= -\frac{1}{2} \left( {\boldsymbol \Sigma}^{-1} - {\boldsymbol \Sigma}^{-1} \left( {\boldsymbol y} - {\boldsymbol \mu} \right) \left( {\boldsymbol y} - {\boldsymbol \mu} \right)' {\boldsymbol \Sigma}^{-1} \right) \end{align} and \begin{align} \nabla_\mu L = \frac{ \partial {\bf L} }{ \partial {\boldsymbol \mu}} &= {\boldsymbol \Sigma}^{-1} \left( {\boldsymbol y} - {\boldsymbol \mu} \right) \end{align} where $\boldsymbol{y}$ are the training samples and $\boldsymbol{L}$ the log likelihood of the multivariate gaussian distribution given by $\mu$ and $\Sigma$. I'm setting a learning rate $\alpha$ and proceed in the following way:

1. Sample an $y$ from unknown $p_\theta(y)$.
2. Compute gradients $\nabla_\Sigma L$ and $\nabla_\mu L$
3. Update: $\mu = \mu + \nabla_\mu L \alpha$, and $\Sigma = \Sigma + \nabla_\Sigma L \alpha$

I've set $\alpha = 0.002$ and $p_\theta(y) = \mathcal{N}(y | 0, 1)$, but it is not convering at all. What am i missing?

P.S.: I know there are different ways, but i'm trying to this using a gradient ascent methood.