# Is the posterior distribution on means in a Bayesian Gaussian mixture model with symmetric priors Gaussian?

I am reading through a document on learning Gaussian mixture models in Infer.NET. They assume the data is generated from 2 Gaussians where the prior distribution on means is Gaussian and the prior distribution on precisions is a Whishart distribution. The prior distribution on the mixture is a Dirichlet distribution. All of these priors are symmetric in the two Gaussians.

They do some inference on some data, and they get back that the posterior distribution on each of the two means is the same Gaussian. They then go on to talk about how to break the symmetry in the model so that the means can converge to different Gaussians.

How can it possibly be that the posteriors on the means are Gaussian? If I observe a million samples from a Gaussian Mixture Model (say unbeknownst to me the data is created by choosing with equal probability a normal distribution of mean 0 and variance 1 or a normal distribution with mean 100 and variance 1) it should be ABSOLUTELY CLEAR what the two means and standard deviations are. The symmetry of course means that the model doesn't know whether the first or the second Gaussian has mean 0 or mean 100, so shouldn't the posterior have two peaks, one near 0 and one near 100? If so, it's obviously not Gaussian.

I would appreciate any help in this matter.

• TLDR; Could this be possibly related: stats.stackexchange.com/questions/178321/… ? – Tim Nov 3 '15 at 14:56
• @time : Yes. It is exactly the label-switching problem I am talking about. Reading through the comments and papers in the link you gave makes it clear that the posteriors in this situation can have multiple modes and so, in general are not Gaussian. In my opinion, this exposes a bug in Infer.NET. – Josh Brown Kramer Nov 3 '15 at 17:18
• TLDR: there is probably a misreading in here. It is well known that the posteriors on individual mixture components can have conjugate priors (including gaussians). – conjectures Jun 27 '17 at 11:31