I am wondering what the predictive distribution of a Dirichlet-Multinomial distribution is.

In this tutorial (page 24), the predictive density is simple and something like "pseudo samples."

enter image description here

However, in this writeup, the predictive density is rather complicated.

enter image description here

I can walk through the latter formula, but do not come to the former one. (It does work from my side for multinoulli/categorical distributions, but not multinomial with N>2).

Can anyone give some references? Thanks a lot.


The first slide applies the general result of the second slide to a case when $$y=\overbrace{(0,\ldots,0,1,0,\ldots,0)}^{1\text{ at }\kappa\text{th position}}$$ with different notations, since [in the second slide] $\alpha'_j=\alpha_j+n_j$ and $y^{(j)}=n_j\in\{0,1\}$. Then \begin{align*} \dfrac{\Gamma(n+1)}{\prod_{j=1}^K \Gamma(y^{(j)}+1)}&=\dfrac{\Gamma(n+1)}{\Gamma(y^{(\kappa)}+1)}=\dfrac{n!}{1}\\ \dfrac{\prod_{j=1}^K \Gamma(y^{(j)}+\alpha_j')}{\prod_{j=1}^K\Gamma(\alpha_j')}&=\dfrac{\Gamma(1+\alpha_\kappa')}{\Gamma(\alpha_\kappa')}=\alpha_\kappa+n_\kappa\\ \end{align*} and the middle ratio does not depend on $y$. Hence $$\mathbb{P}(y=(0,\ldots,0,1,0,\ldots,0))\propto \alpha_\kappa+n_\kappa$$which normalises as $$\mathbb{P}(y_\kappa=1)=\dfrac{\mathbb{P}(y_\kappa=1)}{\sum_{k=1}^K \mathbb{P}(y_k=1)}=\dfrac{ \alpha_\kappa+n_\kappa}{\sum_{k=1}^K \{\alpha_k+n_k\}}=\dfrac{ \alpha_\kappa+n_\kappa}{\sum_{k=1}^K \alpha_k+n}$$

  • $\begingroup$ Thank you very much for answering. If I am understanding correctly, the first slide works with multinomial distributions where N=1, i.e., a categorical distributions. Am I right? $\endgroup$ – Mou Oct 25 '17 at 13:30
  • $\begingroup$ May I make it more clear? The first slide, which I was confused about, predicts the density for a single (N=1) next draw (although the training samples are multinomial distributions with N which can be greater than 2.) Am I right now? $\endgroup$ – Mou Oct 25 '17 at 16:41
  • $\begingroup$ Hi Xi'an, I am currently having trouble in thinking about conjugate prior for Dirichlet processes. Could you please, perhaps, share some quick sort on this? It would be much appreciated. $\endgroup$ – Mou Oct 30 '17 at 22:26
  • $\begingroup$ Well, I'm mainly in engineering background, so if that's too complicated for me, I may feel like skipping that. $\endgroup$ – Mou Oct 30 '17 at 22:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.