1
$\begingroup$

In Appendix C of a paper by Michael E. Tipping and Christopher M. Bishop about mixture models for probabilistic PCA, the probability of a single data vector $\mathbf{t}$ is expressed as a mixture of PCA models (equation 69):

$$ p(\mathbf{t}) = \sum_{i=1}^M\pi_i p(\mathbf{t}|i) $$

where $\pi$ is the mixing proportion and $p(\mathbf{t}|i)$ is a single probabilistic PCA model.

The model underlying the probabilistic PCA method is (equation 2)

$$ \mathbf{t} = \mathbf{Wx} + \boldsymbol\mu + \boldsymbol\epsilon. $$ Where $\mathbf{x}$ is a latent variable. By introducing a new set of variables $z_{ni}$ "labelling which model is responsible for generating each data point $\mathbf{t}_n$", Bishop formulates the complete log likelihood as (equation 70):

$$ \mathcal{L}_C = \sum_{n=1}^N\sum_{i=1}^Mz_{ni}ln\{\pi_ip(\mathbf{t}_n, \mathbf{x}_{ni})\}. $$ I would like to understand how he derives this expression as he doesn't provide a solution himself. How is this expression for the complete log likelihood found?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.