# Compute membership probabilities in E-step of EM algorithm with log-densities instead of densities

As an exercise I have implemented the EM algorithm for Gaussian mixtures, however, I have the problem that in high dimensions the densities of data points become so small that I get a numerical underflow.

Specifically, I am referring to calculating the "membership probabilities" in the E-step (called $$T_{i,j}^{(t)}$$ in this wikipedia article on the EM-algorithm). To obtain the membership probabilities of a data point I need to divide the density from each component by the sum of the densities from all components. For two mixture components and some fixed data point, I therefore need

$$p_i = \frac{\pi_i d_i}{\pi_1 d_1 + \pi_2 d_2},$$

where $$i \in \{1,2\}$$, $$p_i$$ is the membership probability of the data point at hand, $$d_i$$ is the density of the data point at hand, given the distribution of component $$i$$ with current estimates, and $$\pi_i$$ is the current estimate of the mixture proportion of component $$i$$.

Usually, when dealing with numerical underflow I take the log densities. However, then I don't see how to get back to the ratio $$p_i$$, because I can't get $$\log(a + b)$$ from $$\log(a)$$ and $$\log(b)$$.

I'm sure there must be a solution to this, since mixture models are commonly used in high dimensional settings. Can anybody help?

• $$p_1={1}\Big/{1+\exp\{\log \pi_2 +\log d_1-\log \pi_1-\log d_1\}}$$ – Xi'an Apr 2 at 14:08