# Does variational inference solve the label switching problem in Bayesian mixture estimation?

This paper (p. 1751) claims that variational methods do not suffer the label switching problem inherent to Bayesian estimation of mixture models. However, I struggle to find additional references and it is not clear to me why is so.

• Hopefully fixed, thanks for the heads up May 6, 2021 at 19:29
• Concerning the part of your question where you state you struggle to find additional references. The paper states "VB does not suffer from the label-switching problem when fitting mixture models (see Celeux et al. 2000)". Have you looked into the reference in the quotation? May 6, 2021 at 19:43
• Sure. That paper refers to the label switching problem itself, nothing to do with VB May 6, 2021 at 19:45
• @microhaus: in this paper of ours, we do not show that VB does not suffer from the problem, do not cover VB at all, but instead propose solutions to overcome the label switching problem within the genuine posterior distribution. May 6, 2021 at 19:49
• @Xi'an. Thank you for clarifying this. I don't have intimate knowledge of the paper, rather wanted to exhaust the possibility that econ86 had omitted checking the reference. May 6, 2021 at 19:58

The standard VB approximation to the mixture posterior as in McGrory and Titterington (2007) is using conjugate distributions

• $$(\omega_1,\ldots,\omega_k)\sim \mathcal D(\alpha_1,\ldots,\alpha_k)$$
• $$\mu_i|\sigma_is\sim\mathcal N(\xi_i,\delta_i\sigma_i^2)$$
• $$\sigma_i^{-2}\sim\mathcal G(\alpha_i,\beta_i)$$

and the hyperparameters in these distributions are optimised to bring the approximation to be as close as possible to the true posterior. Label switching$$¹$$ means that any permutation of the indices does not change the likelihood, but VB being an optimisation technique, once iterations have started they will not switch to a permutation of the present hyperparameters, just like EM converges to a particular mode of the likelihood, depending on the choice of the starting value.

$$¹$$The reference to our paper (Celeux et al., 2000) is intended to point out the label switching difficulties, not to show that VB does not suffer from label switching.

• I see. But if the hyperparameters are not optimized and for instance set to exchangeable priors (e.g. $\alpha_1 = \dots = \alpha_k$), are we under the same type of unidentifiability even if conducting VB? May 6, 2021 at 19:49
• In my understanding, VB is always optimised within the chosen family, in order to minimise the approximation error. Picking an arbitrary value for the hyperparameters does not seem to make sense. May 6, 2021 at 19:51
• Shouldn't its importance be relatively minor with a lot of data, just as priors in standard Bayesian settings? (Also, could you please point to a reference on optimizing the hyperparameters?) May 6, 2021 at 19:54
• The paper you cited does exactly that. And the whole VB literature. May 6, 2021 at 20:41