2
$\begingroup$

I am going through the paper provided here http://www.cs.cmu.edu/~dgovinda/pdf/recog/EM_algorithm-1.pdf

I could not make out how the following was derived

$\sum_z \mathcal P(\mathbf z|X, \theta_n) \ln \big( \frac{\mathcal P(X| \mathbf z, \theta)\mathcal P(z |\theta)}{\mathcal P(\mathbf z|X, \theta_n)}\big ) - \ln\mathcal P(X|\theta_n)$

=$\sum_z \mathcal P(\mathbf z|X, \theta_n) \ln \big( \frac{\mathcal P(X| \mathbf z, \theta)\mathcal P(z |\theta)}{\mathcal P(\mathbf z|X, \theta_n) \mathcal P(X|\theta_n)}\big ) $

Considering that the left summation in first equation has several terms how is $\ln \mathcal P(X|\theta_n)$ distributed over it?

$\endgroup$
  • $\begingroup$ I think there might be a typo if you check with formula (10) $\endgroup$ – Deep North Oct 20 '15 at 11:58
  • $\begingroup$ i think we need to apply distribution before Jensen's inequality $\endgroup$ – Curious Oct 20 '15 at 12:11
4
$\begingroup$

Isn't that simply that $$\sum_z\mathcal P(z|X,θ_n)=1$$ and hence that \begin{align*} \sum_z &\mathcal P(\mathbf z|X, \theta_n) \ln \left( \frac{\mathcal P(X| \mathbf z, \theta)\mathcal P(z |\theta)}{\mathcal P(\mathbf z|X, \theta_n)}\right ) - \ln\mathcal P(X|\theta_n)\\ &= \sum_z \mathcal P(\mathbf z|X, \theta_n) \ln \left( \frac{\mathcal P(X| \mathbf z, \theta)\mathcal P(z |\theta)}{\mathcal P(\mathbf z|X, \theta_n)}\right ) - \sum_z \mathcal P(\mathbf z|X, \theta_n) \ln\mathcal P(X|\theta_n)\\&=\sum_z \mathcal P(\mathbf z|X, \theta_n) \ln \left( \frac{\mathcal P(X| \mathbf z, \theta)\mathcal P(z |\theta)}{\mathcal P(\mathbf z|X, \theta_n) \mathcal P(X|\theta_n)}\right) \end{align*}

$\endgroup$
  • $\begingroup$ Would be useful to add that this works because ln P(X|theta_n) doesn't have z and can be put outside of sum $\endgroup$ – maximus Jul 18 at 9:36
  • $\begingroup$ Not really..... $\endgroup$ – Xi'an Jul 18 at 14:25
  • $\begingroup$ Then how the sum is same as without it? Could you elaborate? $\endgroup$ – maximus Jul 19 at 10:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.