4
$\begingroup$

I am wondering what the predictive distribution of a Dirichlet-Multinomial distribution is.

In this tutorial (page 24), the predictive density is simple and something like "pseudo samples."

enter image description here

However, in this writeup, the predictive density is rather complicated.

enter image description here

I can walk through the latter formula, but do not come to the former one. (It does work from my side for multinoulli/categorical distributions, but not multinomial with N>2).

Can anyone give some references? Thanks a lot.

$\endgroup$

1 Answer 1

6
$\begingroup$

The first slide applies the general result of the second slide to a case when $$y=\overbrace{(0,\ldots,0,1,0,\ldots,0)}^{1\text{ at }\kappa\text{th position}}$$ with different notations, since [in the second slide] $\alpha'_j=\alpha_j+n_j$ and $y^{(j)}=n_j\in\{0,1\}$. Then \begin{align*} \dfrac{\Gamma(n+1)}{\prod_{j=1}^K \Gamma(y^{(j)}+1)}&=\dfrac{\Gamma(n+1)}{\Gamma(y^{(\kappa)}+1)}=\dfrac{n!}{1}\\ \dfrac{\prod_{j=1}^K \Gamma(y^{(j)}+\alpha_j')}{\prod_{j=1}^K\Gamma(\alpha_j')}&=\dfrac{\Gamma(1+\alpha_\kappa')}{\Gamma(\alpha_\kappa')}=\alpha_\kappa+n_\kappa\\ \end{align*} and the middle ratio does not depend on $y$. Hence $$\mathbb{P}(y=(0,\ldots,0,1,0,\ldots,0))\propto \alpha_\kappa+n_\kappa$$which normalises as $$\mathbb{P}(y_\kappa=1)=\dfrac{\mathbb{P}(y_\kappa=1)}{\sum_{k=1}^K \mathbb{P}(y_k=1)}=\dfrac{ \alpha_\kappa+n_\kappa}{\sum_{k=1}^K \{\alpha_k+n_k\}}=\dfrac{ \alpha_\kappa+n_\kappa}{\sum_{k=1}^K \alpha_k+n}$$

$\endgroup$
4
  • $\begingroup$ Thank you very much for answering. If I am understanding correctly, the first slide works with multinomial distributions where N=1, i.e., a categorical distributions. Am I right? $\endgroup$
    – Mou
    Oct 25, 2017 at 13:30
  • $\begingroup$ May I make it more clear? The first slide, which I was confused about, predicts the density for a single (N=1) next draw (although the training samples are multinomial distributions with N which can be greater than 2.) Am I right now? $\endgroup$
    – Mou
    Oct 25, 2017 at 16:41
  • $\begingroup$ Hi Xi'an, I am currently having trouble in thinking about conjugate prior for Dirichlet processes. Could you please, perhaps, share some quick sort on this? It would be much appreciated. $\endgroup$
    – Mou
    Oct 30, 2017 at 22:26
  • $\begingroup$ Well, I'm mainly in engineering background, so if that's too complicated for me, I may feel like skipping that. $\endgroup$
    – Mou
    Oct 30, 2017 at 22:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.