# Distribution function or density in Mixed Distribution EM

In this calculation

https://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm#E_step

a probability, $P(Z_i = j|X_i=x_i;\theta^{(t)})$ is evaluated using bayes theorem, and then each of the probabilites is written as densities (as opposed to the integrals of densities, which probabilites are).

I should clarify. We have

$$P(Z_i = j|X_i=x_i;\theta^{(t)}) = \frac{P(X_i=x_i|Z_i = 1,\theta^{(t)})P(Z_i = 1)}{P(X_i|\theta^{(t)})} = \frac{f(x_i,\mu_j,\Sigma_j) \tau_j}{f(x_i,\mu_1,\Sigma_1) \tau_1 + f(x_i,\mu_2,\Sigma_2) \tau_2}$$

which implies that $$P(X_i=x_i|Z_i = 1,\theta^{(t)}) = f(x_i,\mu_j,\Sigma_j)$$ I feel like it should rather be $$P(X_i=x_i|Z_i = 1,\theta^{(t)}) = \int_{-\infty}^{x_i} f(s,\mu_j,\Sigma_j)ds$$

So why is the probability equal to the density, and not the integral of the density?

The conditional probabilities $\mathbb{P}(Z_i=j|X_i,\theta^{(t)})$ appear in the E-step of the EM algorithm because the E-step purpose is to produce the expectation of the complete(d) log-likelihood $\log\,L(\theta;x,z)$ conditionally on the observables $(x_1,\ldots,x_n)$. Hence \begin{align*} \overbrace{\mathbb{E}_{Z|X=x,\theta^{(t)}}}^{\text{i.e., expectation}\\\text{conditional on}\\X=x\ \text{and for the}\\\text{value $\theta^{(t)}$ of $\theta^{(t)}$}}\Big[\log\,\overbrace{L(\theta;x,Z)}^{\text{joint density of}\\ \text{sample of }(X_i,Z_i)}\Big] &= \mathbb{E}_{Z|X=x,\theta^{(t)}}\overbrace{\left[\log\,\prod_{i=1}^n L(\theta;x_i,Z_i)\right]}^{\text{independence of the pairs}\\(X_i,Z_i)\ \text{for}\ i=1,\ldots,n}\\ &= \sum_{i=1}^n \mathbb{E}_{Z|X=x,\theta^{(t)}}\left[\log\, L(\theta;x_i,Z_i)\right]\\ &= \sum_{i=1}^n \mathbb{E}_{Z_i|X_i=x_i,\theta^{(t)}}\left[\log\, L(\theta;x_i,Z_i)\right]\\ &= \sum_{i=1}^n \underbrace{\sum_{j=1}^k \mathbb{P}(Z_i=j|X_i=x_i,\theta^{(t)})\log\, L(\theta;x_i,j)}_{\text{definition of conditional expectation}} \end{align*} Note that there is a typo in the line before last in the Wikipedia derivation of $Q(\theta|\theta^{(t)})$, which should be $$\sum_{i=1}^n \sum_{j=1}^2 \mathbb{P}(Z_i=j|X_i=x_i,\theta^{(t)})\log\, \overbrace{L(\theta;x_i,j)}^{\text{since Z_i=j}\\\text{and \theta remains}\\\text{a free variable}}$$rather than$$\sum_{i=1}^n \sum_{j=1}^2 \mathbb{P}(Z_i=j|X_i=x_i,\theta^{(t)})\log\, L(\theta_j;x_i,z_i)$$ Or, with the original notations there, $$\sum_{i=1}^n \sum_{j=1}^2 P(Z_i =j | X_i = \mathbf{x}_i; \theta^{(t)}) \log L(\theta;\mathbf{x}_i,j) \\$$rather than $$\sum_{i=1}^n \sum_{j=1}^2 P(Z_i =j | X_i = \mathbf{x}_i; \theta^{(t)}) \log L(\theta_j;\mathbf{x}_i,\mathbf{z}_i) \\$$ As for the question about using the density and not the cdf in the likelihood, this is a generic one. The likelihood of the pairs $(X_i,Z_i)$ is the product of the densities, by definition of the likelihood function. The notation $$P(X_i=x_i|Z_i = 1,\theta^{(t)})$$ is incorrect when $X_i$ is a continuous random variable, but is not to be confused with $$P(X_i\le x_i|Z_i = 1,\theta^{(t)})$$