# Gibbs sampling with Dirichlet Likelihood

I have a sequence of observations that I am representing as proportions:

 X1    X2    X3    X4    X5
0.10  0.20  0.50  0.12  0.08
0.07  0.24  0.55  0.04  0.10
...


The data are coming in as part of a more complicated hierarchical model, for which I need to perform Gibbs sampling. I have all the other conditionals identified, but I do not know how to choose a conjugate prior nor calculate the posterior for a Dirichlet likelihood.

I've found this paper which reformulates the Dirichlet as an exponential family member and determines the conjugate prior with posterior calculation. However, I'm not familiar with exponential families and it's never really spelled out what the prior is in a non-exponential family form nor what the posterior should be.

Edit: Based on this paper (equation 8), the conjugate prior for a Dirichlet is:

$$p(\vec{a}_j) = f(v, \lambda) \left[\frac{\Gamma (\Sigma_{l=1}^D a_{jl})}{\prod_{l=1}^D \Gamma(a_{jl})}\right]^v \prod_{l=1}^D e^{-\lambda_l(a_{jl}-1)}$$

where $f(v, \lambda)$ is a normalization coefficient. Since $f(v, \lambda)$ is intractable to compute, the best you can get is proportionality. I'm fine with that since I'm only interested in sampling-- it's just not clear to me how one would sample from a member of the above family.

• Please clarify the exact model you have, as it's not clear given your description why you need a prior on the Dirichlet. – jerad Mar 4 '14 at 18:15
• That said, you can always just chose a prior and and sample that conditional posterior using a metropolis step within your gibbs sampler. – jerad Mar 4 '14 at 18:17
• The exact model that I have is that each data point $X$ is a sample from a $Dirichlet(\vec{\alpha})$ with unknown $\vec{\alpha}$. I could use a MH step, but it's unclear what a good proposal distribution would be. – Wesley Tansey Mar 4 '14 at 20:53

## 1 Answer

The conditionals of the posterior for the Dirichlet likelihood model are not of standard form so (as far as I know) you can not write a straightforward Gibbs sampler. You can either

1. Use a different MCMC sampling procedure, such as Metropolis-Hastings or a slice sampler.
2. Use a different likelihood, such as a constrained Gaussian model where it is possible to write a Gibbs sampler.