An extended version of log-likelihood for a multivariate Gaussian mixture model (e.g., get rid of the log before sum?) As you know, the likelihood of a multivariate Gaussian mixture model with C components and $d$ dimensional $x$ is as follow:
$$p(x|\lambda) = \Sigma^{C}_{c=1}w_c p(x_t|\mu_c,\Sigma_c)$$
where $\lambda=\{w,\mu, \Sigma\}$, $w_c$ is the weight for component $c$ and $p(x_t|\mu_c,\Sigma_c)$ is  Gaussian probability density function with mean $\mu_c$ and covariance matrix $\Sigma_c$.
consequently, the log-likelihood would be:
$$log(p(x|\lambda))=log(\Sigma^{C}_{c=1}w_c p(x_t|\mu_c,\Sigma_c))$$
now my question:
is there any way to get rid of the log before the sum?
It causes some numerical instabilities because of the exponential in the normal PDF, when the covariance matrix becomes very small.
If log taken inside the sum, then:$$log(exp(X)) = X$$
I am looking for a solution for the general case for a $d$ dimensional $x$ and $C$ components GMM, with the PDF as follow:

Any ideas?
 A: Switching logs and sums is not a possibility. When considering$$\sum_{t=1}^t \log\left[ \sum_{c=1}^C w_c p(x_t|\mu_c,\Sigma_c) \right]$$from an overflow-underflow perspective, you could


*

*ascertain whether or not exploring such extreme values for your parameters makes sense for the problem at hand. If not, treat extreme values as producing zero likelihoods;

*check if any of the determinants $|\Sigma_c|$ creates an indeterminacy issue and if so treat the determinant aside from the rest of the density by computing directly the log;

*determine which term $w_c p(x_t|\mu_c,\Sigma_c)$ is largest among all pairs $(t,c)$. For this you can look directly at the logarithms since$$\arg\max_{t,c} w_c p(x_t|\mu_c,\Sigma_c) =\arg\max_{t,c} \log\{w_c p(x_t|\mu_c,\Sigma_c)\}$$If this maximum creates an indeterminacy, then treat the whole likelihood as zero;

*remove this maximal value from all terms and treat underflow terms as zeros. If a given observation $x_t$ is such that all (log-)terms $\log\{w_c p(x_t|\mu_c,\Sigma_c)\}$ are indeterminate, then treat the whole likelihood as zero;

*check this impressive answer on Cross validated about the approximation of a Gaussian tail density by W. Huber if the problem persists.

