2
$\begingroup$

Problem: Show that the following distributions are conjugate priors for the corresponding densities..

The multinomial distribution with $k$ categories and

$$ p_{X|\theta_1 , \dots, \theta_k} (x_1, \dots, x_k) = \frac{n!}{x_1! \cdot \dots \cdot x_k!} \prod_{i=1}^{k} \theta_i^{x_i}$$

where $x_i \geq 0 $ are integers and $\sum_{i=1}^{k} x_i = n$ has the Dirichlet distribution,

$$ P_{\Theta_1, \dots, \Theta_k} (\theta_1, \dots, \theta_k) = \frac{\prod_{i=1}^{k} \Gamma(\alpha_i)}{\Gamma(\sum_{i=1}^{k} \alpha_i)} \prod_{i=1}^{k} \theta_i^{\alpha_i-1}$$

Attempted solution:

Need to calculate the posterior $p_{\Theta | X} = \frac{p_{X|\Theta} p_{\Theta}}{p_X}$ and see that it is a Dirichlet distributions for some hyperparameters $\beta_i$.

The numerator is

$$ p_{X|\Theta} p_{\Theta} = \frac{n!}{x_1! \cdot \dots \cdot x_k!} \frac{\prod_{i=1}^{k} \Gamma(\alpha_i)}{\Gamma(\sum_{i=1}^{k} \alpha_i)} \prod_{i=1}^{k} \theta_i^{x_i + \alpha_i-1} := \xi \prod_{i=1}^{k} \theta_i^{x_i + \alpha_i-1}$$

And the denominator \begin{equation} \begin{split} p_X &= \int_{\theta} p_{X|\Theta} p_{\Theta} d\Theta = \xi \prod_{i=1}^{k} \int_{0}^{1} \theta_i^{x_i + \alpha_i-1} d\theta_i \\ &= \xi \prod_{i=1}^{k} [\frac{\theta_i^{x_i + \alpha_i}}{x_i + \alpha_i} ]_{\theta = 0}^{1} \end{split} \end{equation}

Any hints...?

Best regards

$\endgroup$
1
  • $\begingroup$ Your last equation solving the integral is not right. The integration is done on the surface $\sum \theta_i = 1$, and you have been solving it as if the integral is on the cube where each $\theta_i$ is from 0 to 1. $\endgroup$ Nov 10, 2023 at 17:26

1 Answer 1

1
$\begingroup$

The numerator is

$$ p_{X|\Theta} p_{\Theta} = \xi \prod_{i=1}^{k} \theta_i^{x_i + \alpha_i-1}$$

This means that

$$\frac{p_{X|\Theta} p_{\Theta}}{p_X} \propto \prod_{i=1}^{k} \theta_i^{x_i + \alpha_i-1} $$

For practical purposes you do not really have to deduce the normalisation constant because you already know that the formula is of the form $\prod_{i=1}^{k} \theta_i^{x_i + \alpha_i-1}$, which is a Dirichlet distribution.

If you are required to compute as well the normalisation constant because this is some homework exercise, then you could check out the relationship between the beta function and the gamma function https://en.m.wikipedia.org/wiki/Beta_function#Relationship_to_the_gamma_function

$\endgroup$
2
  • $\begingroup$ Fair enough; $p_X$ won't cancel out that term. Thanks! $\endgroup$
    – Oskar
    Nov 10, 2023 at 16:49
  • $\begingroup$ @Oskar the important thing is that the term is independent of the $\theta_i$, so it will be just a constant. $\endgroup$ Nov 10, 2023 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.