# How to diagnose HMC results like r-hat for a Mixture Model?

I have the following distribution \begin{align} \boldsymbol \pi&\sim\text{Dirichlet}([1,\cdots 1]\in R^K)\\ \boldsymbol \theta&\sim P(\boldsymbol \theta) \\ \mathbf y&\sim \sum _{i=1}^K\pi_i\text{Joint}(\theta_k) \end{align} This is a Bayesian Mixture Model specified by the mixing weights $$\pi_i$$, paramaters $$\theta_i$$, and number of components $$K$$.

After sampling 5 chains from an HMC, the samples for $$\boldsymbol \pi\in R^{5\times K}$$ is not identifiable. In other words, if there are three dominant values for $$\pi_i,\pi_j,\pi_k$$ when $$K=5$$, these three dominants would be jumbled in different combinations.

This would cause MCMC diagnostics like r hat to give wrong values. In fact, MCMC diagnostics would give a nearly uniform distribution as the mean of the HMC sampled $$\boldsymbol \pi$$.

How do I perform diagnostics? Do i simply run a single but very large number of chain samples and split these ?

For example, sample $$1000$$ samples using an HMC sampler and split these into five $$200$$ samples and pass these into a MCMC diagnostics function?

• Have you looked up the 'label switching problem' for mixture models? Sep 4 at 7:25

This is only a partial answer, but in general you can go a long way by adding an order constraint to your model: enforcing $$\theta_1 \lt \theta_2 \lt\dots\lt \theta_k$$. This is trickier to do if $$\theta$$ is multidimensional (forum).