4
$\begingroup$

I am going through the paper provided here http://www.cs.cmu.edu/~dgovinda/pdf/recog/EM_algorithm-1.pdf

I could not make out how the following was derived

$\sum_z \mathcal P(\mathbf z|X, \theta_n) \ln \big( \frac{\mathcal P(X| \mathbf z, \theta)\mathcal P(z |\theta)}{\mathcal P(\mathbf z|X, \theta_n)}\big ) - \ln\mathcal P(X|\theta_n)$

=$\sum_z \mathcal P(\mathbf z|X, \theta_n) \ln \big( \frac{\mathcal P(X| \mathbf z, \theta)\mathcal P(z |\theta)}{\mathcal P(\mathbf z|X, \theta_n) \mathcal P(X|\theta_n)}\big ) $

Considering that the left summation in first equation has several terms how is $\ln \mathcal P(X|\theta_n)$ distributed over it?

$\endgroup$
2
  • $\begingroup$ I think there might be a typo if you check with formula (10) $\endgroup$
    – Deep North
    Oct 20, 2015 at 11:58
  • $\begingroup$ i think we need to apply distribution before Jensen's inequality $\endgroup$
    – Curious
    Oct 20, 2015 at 12:11

1 Answer 1

4
$\begingroup$

Isn't that simply that $$\sum_z\mathcal P(z|X,θ_n)=1$$ and hence that \begin{align*} \sum_z &\mathcal P(\mathbf z|X, \theta_n) \ln \left( \frac{\mathcal P(X| \mathbf z, \theta)\mathcal P(z |\theta)}{\mathcal P(\mathbf z|X, \theta_n)}\right ) - \ln\mathcal P(X|\theta_n)\\ &= \sum_z \mathcal P(\mathbf z|X, \theta_n) \ln \left( \frac{\mathcal P(X| \mathbf z, \theta)\mathcal P(z |\theta)}{\mathcal P(\mathbf z|X, \theta_n)}\right ) - \sum_z \mathcal P(\mathbf z|X, \theta_n) \ln\mathcal P(X|\theta_n)\\&=\sum_z \mathcal P(\mathbf z|X, \theta_n) \ln \left( \frac{\mathcal P(X| \mathbf z, \theta)\mathcal P(z |\theta)}{\mathcal P(\mathbf z|X, \theta_n) \mathcal P(X|\theta_n)}\right) \end{align*}

$\endgroup$
4
  • $\begingroup$ Would be useful to add that this works because ln P(X|theta_n) doesn't have z and can be put outside of sum $\endgroup$
    – maximus
    Jul 18, 2019 at 9:36
  • $\begingroup$ Not really..... $\endgroup$
    – Xi'an
    Jul 18, 2019 at 14:25
  • $\begingroup$ Then how the sum is same as without it? Could you elaborate? $\endgroup$
    – maximus
    Jul 19, 2019 at 10:00
  • $\begingroup$ I believe it is due to the law of total probability $\endgroup$
    – Jayyu
    Sep 5, 2022 at 14:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.