Suppose $x \sim \text{Multinomial}(p)$ and $p \sim \text{Dirichlet}(\mathbf{1})$ (i.e. the flat Dirichlet distribution). What is

$$\mathrm{P}(x) = \int \mathrm{P}(x, p) \,\mathrm{d}p = \int \mathrm{P}(p) \mathrm{P}(x \mid p) \,\mathrm{d}p$$

i.e. the marginal likelihood or model evidence of $x$ under a multinomial distribution?

Recall that

$$\mathrm{P}(x \mid p) = \left( \sum_i x_i \right)! \prod_i \frac{p_i^{x_i}}{x_i!}$$

Hence, letting $S$ be the simplex satisfying $\mathrm{P}(p) \neq 0$:

\begin{align*} \mathrm{P}(x) &= \int \mathrm{P}(p) \left( \sum_i x_i \right)! \prod_i \frac{p_i^{x_i}}{x_i!} \,\mathrm{d}p \\ &= \mathrm{P}(p) \int_S \left( \sum_i x_i \right)! \prod_i \frac{p_i^{x_i}}{x_i!} \,\mathrm{d}p \\ &= \mathrm{P}(p) \left( \sum_i x_i \right)! \int_S \prod_i \frac{p_i^{x_i}}{x_i!} \,\mathrm{d}p \\ &= \mathrm{P}(p) \left( \sum_i x_i \right)! \prod_i \frac{1}{x_i!} \int_S p_i^{x_i} \,\mathrm{d}p \\ &= \mathrm{P}(p) \left( \sum_i x_i \right)! \prod_i \frac{1}{x_i!} \int_{0 < p_1 < 1} \int_{0 < p_2 < 1 - p_1} \ldots \int_{p_n = 1 - (p_1 + \ldots p_{n-1})} p_i^{x_i} \,\mathrm{d}p_1 \mathrm{d}p_2 \ldots \mathrm{d}p_n \\ &= \mathrm{P}(p) \left( \sum_i x_i \right)! \prod_i \frac{1}{x_i!} \int_{0 < p_i < 1} \int_{\sum_{j \neq i} p_j = 1} p_i^{x_i} \prod_{j \neq i} \,\mathrm{d}p_j \mathrm{d}p_i \\ &= \mathrm{P}(p) \left( \sum_i x_i \right)! \prod_i \frac{1}{x_i!} \int_{0 < p_i < 1} p_i^{x_i} \left( \int_{\sum_{j \neq i} p_j = 1 - p_i} \prod_{j \neq i} \,\mathrm{d}p_j \right) \mathrm{d}p_i \\ &= \mathrm{P}(p) \left( \sum_i x_i \right)! \prod_i \frac{1}{x_i!} \int_{0 < p_i < 1} p_i^{x_i} (1 - p_i) (1 - (p_i + p_2)) \ldots \mathrm{d}p_i \\ \end{align*}

How can I simplify this expression further?

  • $\begingroup$ What exactly is $P$ supposed to be? It's not a density function because it is not normalized to integrate to unity. $\endgroup$ – whuber Oct 3 '16 at 0:47
  • $\begingroup$ @whuber Are you referring to $\mathrm{P}(p)$? $\endgroup$ – user76284 Oct 3 '16 at 0:49
  • $\begingroup$ It's true--you are using "$P$" in at least three different senses. I was referring to the first definition. $\endgroup$ – whuber Oct 3 '16 at 0:50
  • $\begingroup$ @whuber I think I know what you mean. I meant to use a flat Dirichlet distribution for the prior for $p$. $\endgroup$ – user76284 Oct 3 '16 at 0:51
  • 1
    $\begingroup$ OK. Your work would simplify considerably by ignoring all normalization constants anyway. Doing so might help you see the essence of the problem. $\endgroup$ – whuber Oct 3 '16 at 1:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.