3
$\begingroup$

I am studying EM-algorithm for mixture data. I read that some authors said that, the E-step is equivalent to the calculation of the posterior probability (I think this come from Bayesian rule). So, my question is, is it acceptable to says that the E-step is the same as or can be interpreted as a posterior probability? I meant will I be wrong if I said that the E-step can be viewed as a posterior probability that the point $s$ belongs to the $k^{th}$ component.

$\endgroup$
3
$\begingroup$

It is indeed incorrect in that the EM algorithm usually operates outside the Bayesian paradigm and hence posterior probabilities are not defined there. What is correct is that the E step for mixtures is equivalent to computing the conditional probability of an allocation of observation $i$ to component $j$: $$\mathbb{E}[\mathbb{I}_j(Z_i)|X_i=x_i,\theta]=\mathbb{P}(Z_i=j|X_i=x_i,\theta)=\dfrac{p_j f(x_i|\theta_j)}{\sum_\nu p_\nu f(x_i|\theta_\nu)}$$ This probability is both conditional on $X_i=x_i$ and on the value of the parameter $\theta$. The posterior probability that $Z_i=j$ would be conditional on the sample $(X_1,\ldots,X_n)$ only, integrating $\theta$ out.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.