Symmetry group in posterior distribution/inference

Question

Here's a scenario: Suppose I collect a dataset $\{x_i\}_{i=1}^k\subseteq\mathbb R$ of data points $x_i$, and I wish to explain it using a mixture of two Gaussians; assume the unknown parameters are the means $\mu_1$ and $\mu_2$ of the mixture model. Following Bayesian methodology, I formulate some posterior distribution $P((\mu_1,\mu_2)|\{x_i\}_i)$, from which I can use Monte Carlo methods to sample lots of likely $(\mu_1^j,\mu_2^j)$ pairs.

The problem is, swapping $(\mu_1,\mu_2)$ to $(\mu_2,\mu_1)$ describes the same mixture of Gaussians; a symmetry group---in this case $\mathbb{Z}/2$---acts on the random samples I'm generating without changing their interpretation. This prevents me, for example, from directly averaging the pairs that come out of my sampler. That is, the expectation $\mathbb E_{(\mu_1,\mu_2)\sim P(\cdots)}[(\mu_1,\mu_2)]$ really doesn't make sense since the ordering of the two elements is arbitrary in each sample.

Is there a general mathematical/statistical framework for coping with group-invariant probability measures? What inference algorithms do people use in this scenario? Do notions like mean/covariance extend to quotient spaces?

Any pointers are much appreciated. This seems like a fundamental/classical problem, and likely I'm just missing Google search terms! My apologies if I have abused "Bayesian language" here---this is not my typical research field.

Answer 1

$\qquad\qquad\qquad$

This is a well-documented issue with mixture modelling, although in your case of having only the means unknown, they are identifiable when the weights in the mixture differ from $(1/2,1/2)$. A first branch of results in this area follow the 1963 paper of Teicher. And indeed a plain Bayesian approach under exchangeable prior assumptions cannot handle using a posterior mean since both marginals are identical. Another branch of results attempts at restricting the parameter space to make it identifiable. From strong truncation imposing ordering to representing the posterior as a point process as described in the first chapter by Peter Green in our very recent Handbook of Mixture Analysis, to isolate a single mode in the posterior distribution by fighting label switching, as also discussed in this X validated question, and advocated by Geweke (2007). See also my entries on this topic on my blog.