Newton-Raphson method to solve for dof when performing MLE of a multivariate Student-t distribution using EM

Question

I am reading the derivation of EM algorithm to estimate the maximum likelihood of a multivariate Student-t distribution $\mathcal{T}(\mathbf{x} \vert \pmb{\mu}, \pmb{\Sigma}, \nu)$ in Kevin Murphy's textbook (Machine Learning: a probabilistic perspective, section 11.4.5). I can follow the whole section, but have a concern at the very last step when determining the dof $\nu$.

The E-step for $\nu$ is: \begin{equation} \mathbb{E}\left[L_{G}(\nu) \right] = -N \log \Gamma \left(\frac{\nu}{2} \right) + \frac{N\nu}{2} \log \left(\frac{\nu}{2}\right) + \frac{\nu}{2} \sum_{i=1}^{N} \left( \bar{\ell}^{(t)}_{i} - \bar{z}_{i}^{(t)} \right), \tag{11.77} \label{11.77} \end{equation} where $\Gamma(x)$ is the gamma function.

The M-step is to maximize $\mathbb{E}\left[L_{G}(\nu) \right]$ w.r.t. $\nu$ by solving for the root of the first derivative: \begin{equation} \frac{\partial \mathbb{E}\left[L_{G}(\nu) \right]}{\partial \nu} = -\frac{N}{2} \Psi \left(\frac{\nu}{2} \right) + \frac{N}{2} \log \left(\frac{\nu}{2} \right) + \frac{N}{2} + \frac{1}{2}\sum_{i=1}^{N} \left( \bar{\ell}^{(t)}_{i} - \bar{z}_{i}^{(t)} \right), \tag{11.78} \label{11.78} \end{equation} where $\Psi(x)$ is the digamma function.

As discussed in the textbook, the root of \eqref{11.78} can be found by using some numerical methods for root finding, such as half-interval method. However, I wonder if we can apply Newton-Raphson method for \eqref{11.78} by taking second derivative: \begin{equation} \frac{\partial^2 \mathbb{E}\left[L_{G}(\nu) \right]}{\partial\nu^{2}} = -\frac{N}{4} \Psi^{(2)} \left(\frac{\nu}{2} \right) + \frac{N}{2\nu}, \end{equation} where $\Psi^{(2)}(x)$ is the trigamma function.

Given the second derivative, we can apply the Newton-Raphson method to iteratively find the dof $\nu$ as: \begin{equation} \nu \gets \nu - \left(\frac{\partial \mathbb{E}\left[L_{G}(\nu) \right]}{\partial \nu}\right)/\left(\frac{\partial^2 \mathbb{E}\left[L_{G}(\nu) \right]}{\partial\nu^{2}}\right). \end{equation}

My question is if we can apply Newton-Raphson method to find $\nu$. What are the disadvantages of the Newton-Raphson method preventing us from using in this case? I am aware that Newton-Raphson method requires the second derivative, but in this case, the second derivative can be efficiently calculated. Or, I might be wrong because of the complexity of the digamma calculation.

Answer 1

You're correct, the Newton-Raphson method could be used here as well. In fact, you can treat it as a root-finding method applied to the (11.78).

The trigamma function does not introduce challenge either. Surely it's not available in a closed form, but so does digamma, and there're efficient numerical procedures to compute both of them, in fact scipy has a generic scipy.special.polygamma function that covers both of them.