Convergence of EM in Mixture Models w.r.t unlikely events $(f(\cdot)=0)$ in either distribution To maximize the likelihood of a mixture model with unobserved latent variables, the Expectation Maximization is conventionally applied. Assuming we have data $x_1,\dots,x_n$ from a fixed number of distributions, with corresponding density functions $f_j(x;\theta_j)$. Now let the variables $z_{1j},\dots,z_{nj}$ be the binary variable denoting if the corresponding observation is a random outcome of distribution $j$. Now, the full likelihood of observation $i$ is often formulated as follows
$L(\theta;x_i,z)=\prod_{j}f_j(x_i;\theta_j)^{z_{ij}}$ with log-likelihood
$l(\theta;x_i,z)=\sum_{j}{z_{ij}}\ln(f_j(x_i;\theta_j))$
In the E-step of the EM algorithm, the latent variable $z_{ij}$ is estimated w.r.t current parameters. Now, what happens if observation $x_i$ belongs to distribution $j$ and is such that for some other distribution $k\neq j$, $f_k(x_i,\theta_k)=0$. Then the log-likelihood will take value $-\infty$.
I understand $E({z_{ik}}|x_i,\theta_k)$ must take value 0 in this case, and thus "removing" this term from the summation. However, in most programming languages $0\cdot \infty$ is ill defined (I believe, for instance R returns NaN), so won't this create convergence issues? Is there a standard procedure to deal with this? 
I appreciate all comments/references to literature and answers.
 A: For those reading at home who are worried by the whole $0 \times \infty = 0$, it's reasonable to accept that if $z_{ij} = 0$ then $z_{ij} \log (f_j(x_i, \theta_j)) = 0$ beyond the mere conveniece of it. To see this, note that
$
z_{ij} = \frac{f_j(x_i, \theta_j)}{\sum_k f_k(x_i, \theta_k)}
$
and that if we take the limiting value of 
$
\frac{f_j(x_i, \theta_j)}{\sum_k f_k(x_i, \theta_k)} \log (f_j(x_i, \theta_j) )
$
as $f_j(x_i, \theta_j) \rightarrow 0^+$, we get the answer zero* provided there exists at least one other $f_k(x_i, \theta_k) > 0$.
From a programming point of view I'm not aware of any convention on how to deal with this. Some languages will define $0^0 = 1$ even if $0 \log (0)$ is not defined (matlab, for example), so you could evaluate $\log (f_j(x_i, \theta_j)^{z_{ij}})$ instead of  $z_{ij} \log (f_j(x_i, \theta_j))$ and avoid NaNs. This gives nice clean looking code, but is problematic for a number of other reasons (you replace a multiplication with an exponentiation, and I think it can generate precision issues). 
It's best really to just have an If statement that checks whether each $z_{ij} == 0$, and skips evaluating that log likelihood entirely if so. Alternatively, you could evaluate $z_{ij} \log( \max (f_j(x_i, \theta_j), \epsilon) )$ where $\epsilon$ is the smallest value which your logarithm gives finite answers for. If $z_{ij}=0$, you'll get the same result.
*The easiest way to check this is to consider the limit of
$
\frac{\log (a)}{ 1/a}
$
as $a \rightarrow 0$. As both numerator and denominator tend to $\infty$ we can just apply L'Hopital's rule.
