The Dirichlet Process prior is a Bayesian non-parametric prior to model your data as coming from an infinite mixture of distributions. Since your data is finite, only a finite number of these mixture components will be populated.

How is this practically different from a finite mixture model where the number of mixture components is equal to your data? If you put a prior on the mixture weights of this finite mixture, you will learn their posterior. Undoubtedly many of these weights will be tiny and their corresponding mixture will be effectively unpopulated.

A sampler for the finite mixture will be easier to write and won't have the same "mixture label swapping" problem that the the Dirichlet Process will have. The Dirichlet Process must have some other advantage - what am I missing?

  • $\begingroup$ The key difference is, we don't assume K mixing components from the beginning, and K can grow as data arrives. The general idea is to keep model flexible and let it grow its complexity if needed. Take one of Whye Teh's videos and watch. Example: videolectures.net/mlss2011_teh_nonparametrics/?q=why%20teh $\endgroup$ – user3639557 Sep 4 '17 at 3:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.