# Predictive Density for Dirichlet Multinomial

I am wondering what the predictive distribution of a Dirichlet-Multinomial distribution is.

In this tutorial (page 24), the predictive density is simple and something like "pseudo samples."

However, in this writeup, the predictive density is rather complicated.

I can walk through the latter formula, but do not come to the former one. (It does work from my side for multinoulli/categorical distributions, but not multinomial with N>2).

Can anyone give some references? Thanks a lot.

The first slide applies the general result of the second slide to a case when $$y=\overbrace{(0,\ldots,0,1,0,\ldots,0)}^{1\text{ at }\kappa\text{th position}}$$ with different notations, since [in the second slide] $\alpha'_j=\alpha_j+n_j$ and $y^{(j)}=n_j\in\{0,1\}$. Then \begin{align*} \dfrac{\Gamma(n+1)}{\prod_{j=1}^K \Gamma(y^{(j)}+1)}&=\dfrac{\Gamma(n+1)}{\Gamma(y^{(\kappa)}+1)}=\dfrac{n!}{1}\\ \dfrac{\prod_{j=1}^K \Gamma(y^{(j)}+\alpha_j')}{\prod_{j=1}^K\Gamma(\alpha_j')}&=\dfrac{\Gamma(1+\alpha_\kappa')}{\Gamma(\alpha_\kappa')}=\alpha_\kappa+n_\kappa\\ \end{align*} and the middle ratio does not depend on $y$. Hence $$\mathbb{P}(y=(0,\ldots,0,1,0,\ldots,0))\propto \alpha_\kappa+n_\kappa$$which normalises as $$\mathbb{P}(y_\kappa=1)=\dfrac{\mathbb{P}(y_\kappa=1)}{\sum_{k=1}^K \mathbb{P}(y_k=1)}=\dfrac{ \alpha_\kappa+n_\kappa}{\sum_{k=1}^K \{\alpha_k+n_k\}}=\dfrac{ \alpha_\kappa+n_\kappa}{\sum_{k=1}^K \alpha_k+n}$$

• Thank you very much for answering. If I am understanding correctly, the first slide works with multinomial distributions where N=1, i.e., a categorical distributions. Am I right? – Mou Oct 25 '17 at 13:30
• May I make it more clear? The first slide, which I was confused about, predicts the density for a single (N=1) next draw (although the training samples are multinomial distributions with N which can be greater than 2.) Am I right now? – Mou Oct 25 '17 at 16:41
• Hi Xi'an, I am currently having trouble in thinking about conjugate prior for Dirichlet processes. Could you please, perhaps, share some quick sort on this? It would be much appreciated. – Mou Oct 30 '17 at 22:26
• Well, I'm mainly in engineering background, so if that's too complicated for me, I may feel like skipping that. – Mou Oct 30 '17 at 22:33