On my book, "Machine Learning A Probabilistic Approach". It's stated that is straightforward to derive a Gibbs sampling algorithm to fit a mixture model, especially if we use conjugate priors.

So straightforward that book gives an example of fitting Mixture Gaussian without actually giving the resulting fitting algorithm.

Here the example:enter image description here

My question is: once I have all the full conditionals of the discrete indicators, mixing weight, means and covariance, how shold I proceed for actually fitting my data? What is the algorithm that I should follow?

  • $\begingroup$ You should first read the section on Gibbs sampling. $\endgroup$ – Xi'an Jan 11 '19 at 13:29
  • $\begingroup$ I've read it but I can just hunch the prototype of the algorithm, do I need to sample in turn from equation 24.10, 24.11, 24.12, 24.17, using the new samples values each time? $\endgroup$ – Tommaso Bendinelli Jan 11 '19 at 14:03

If I learned this right, you use the MCMC estimates essentially as replacement for the M step, in an EM like process.

The idea is simple: if we cannot solve the maximum likelihood problem exactly, we use anumeric approximation instead.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.