Why choose a Dirichlet distribution in Latent Dirichlet Allocation (LDA)?

Question

I am studying Latent Dirichlet Allocation (LDA), but I don't have much knowledge of stochastic processes.

From Wikipedia:

LDA assumes the following generative process for a corpus $D$ consisting of $M$ documents each of length $N_{i}$:

1. Choose $\theta _{i}\sim \operatorname {Dir} (\alpha )$, where $i\in \{1,\dots ,M\}$ and $\mathrm {Dir} (\alpha )$ is a Dirichlet distribution with a symmetric parameter $\alpha$

My question is this: why Dirichlet distribution? Are there any particular properties of this distribution that make it suitable for this model?

Answer 1

Two properties that make the Dirichlet distribution a suitable default choice are:

1. Flexibility: it has full support on the parameter space which is the set possible values of the K-dimensional vector of probabilities, which must sum to 1 (i.e., the K-1 dimensional simplex)

2. Computational tractability: the Dirichlet distribution is the conjugate prior for the multinomial likelihood and can be analytically integrated out to obtain tractable marginal likelihood and Pólya urn schemes (see Dirichlet multinomial) this allows to have tractable algorithms to perform Bayesian inference (e.g., it gives tractable full conditional distribution in a Gibbs-sampler).

Point 2. is explained with more technical details in the Wikipedia page linked in your question in the Section Aspects of computational details or in the paper: Blei, David M., Ng, Andrew Y. and Jordan, Michael I. "Latent Dirichlet Allocation". Journal of Machine Learning Research (2003)