# Show that the multinomial distribution with $k$ categories and Dirichlet distribution are conjugate prior

Problem: Show that the following distributions are conjugate priors for the corresponding densities..

The multinomial distribution with $$k$$ categories and

$$p_{X|\theta_1 , \dots, \theta_k} (x_1, \dots, x_k) = \frac{n!}{x_1! \cdot \dots \cdot x_k!} \prod_{i=1}^{k} \theta_i^{x_i}$$

where $$x_i \geq 0$$ are integers and $$\sum_{i=1}^{k} x_i = n$$ has the Dirichlet distribution,

$$P_{\Theta_1, \dots, \Theta_k} (\theta_1, \dots, \theta_k) = \frac{\prod_{i=1}^{k} \Gamma(\alpha_i)}{\Gamma(\sum_{i=1}^{k} \alpha_i)} \prod_{i=1}^{k} \theta_i^{\alpha_i-1}$$

Attempted solution:

Need to calculate the posterior $$p_{\Theta | X} = \frac{p_{X|\Theta} p_{\Theta}}{p_X}$$ and see that it is a Dirichlet distributions for some hyperparameters $$\beta_i$$.

The numerator is

$$p_{X|\Theta} p_{\Theta} = \frac{n!}{x_1! \cdot \dots \cdot x_k!} \frac{\prod_{i=1}^{k} \Gamma(\alpha_i)}{\Gamma(\sum_{i=1}^{k} \alpha_i)} \prod_{i=1}^{k} \theta_i^{x_i + \alpha_i-1} := \xi \prod_{i=1}^{k} \theta_i^{x_i + \alpha_i-1}$$

And the denominator $$$$\begin{split} p_X &= \int_{\theta} p_{X|\Theta} p_{\Theta} d\Theta = \xi \prod_{i=1}^{k} \int_{0}^{1} \theta_i^{x_i + \alpha_i-1} d\theta_i \\ &= \xi \prod_{i=1}^{k} [\frac{\theta_i^{x_i + \alpha_i}}{x_i + \alpha_i} ]_{\theta = 0}^{1} \end{split}$$$$

Any hints...?

Best regards

• Your last equation solving the integral is not right. The integration is done on the surface $\sum \theta_i = 1$, and you have been solving it as if the integral is on the cube where each $\theta_i$ is from 0 to 1. Commented Nov 10, 2023 at 17:26

The numerator is

$$p_{X|\Theta} p_{\Theta} = \xi \prod_{i=1}^{k} \theta_i^{x_i + \alpha_i-1}$$

This means that

$$\frac{p_{X|\Theta} p_{\Theta}}{p_X} \propto \prod_{i=1}^{k} \theta_i^{x_i + \alpha_i-1}$$

For practical purposes you do not really have to deduce the normalisation constant because you already know that the formula is of the form $$\prod_{i=1}^{k} \theta_i^{x_i + \alpha_i-1}$$, which is a Dirichlet distribution.

If you are required to compute as well the normalisation constant because this is some homework exercise, then you could check out the relationship between the beta function and the gamma function https://en.m.wikipedia.org/wiki/Beta_function#Relationship_to_the_gamma_function

• Fair enough; $p_X$ won't cancel out that term. Thanks! Commented Nov 10, 2023 at 16:49
• @Oskar the important thing is that the term is independent of the $\theta_i$, so it will be just a constant. Commented Nov 10, 2023 at 16:56