The Dirichlet Process Mixture of Gaussians has been well studied and shown to work. I have never seen DPM of Multinomials and tried to implement one. What I notice is the likelihood tends to dominate the mixture proportion and the algorithm eventually makes every observation a singleton cluster. This does not happen for DPM of Gaussians because the likelihood values are not extreme even when the model is misspecified therefore the mixture proportion is not dominated by likelihood. The purpose of the mixture proportion is to penalize singletons but when the likelihood dominates, it basically does does nothing which explains why every cluster is a singleton. Is there away around this or is DPM of Multinomials just not realistic?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.