15
$\begingroup$

Label switching (i.e., the posterior distribution is invariant to switching component labels) is a problematic issue when using MCMC to estimate mixture models.

  1. Is there a standard (as in widely accepted) methodology to deal with the issue?

  2. If there is no standard approach then what are the pros and cons of the leading approaches to solve the label switching problem?

$\endgroup$
  • $\begingroup$ I was considering asking "How can I do an MCMC model of the output on lmer for models with random slopes?" but I'm wondering whether that question is redundant with this one. That is, is the "label switching issue" when using MCMC to estimate mixture models the same sort of problem take makes it so that pvals.fnc() in languageR is able to MCMC intercept models but not models with slopes? If not, please let me know and I'll go back to asking my initial question. $\endgroup$ – russellpierce Dec 13 '10 at 1:14
  • $\begingroup$ @drknexus I do not know R to comment in your question. Perhaps, you should just post your question with a comment that your qn may be lined to this one. $\endgroup$ – user28 Dec 14 '10 at 4:10
13
$\begingroup$

There is a nice and reasonably recent discussion of this problem here:

Christian P. Robert Multimodality and label switching: a discussion. Workshop on mixtures, ICMS March 3, 2010.

Essentially, there are several standard strategies, and each has pros and cons. The most obvious thing to do is to formulate the prior in such a way as to ensure there is only one posterior mode (eg. order the means of the mixuture components), but this turns out to have a strange effect on the posterior, and therefore isn't generally used. Next is to ignore the problem during sampling, and then post-process the output to re-label the components to keep the labels consistent. This is easy to implement and seems to work OK. The more sophisticated approaches re-label on-line, either by keeping a single mode, or deliberately randomly permuting the labels to ensure mixing over multiple modes. I quite like the latter approach, but this still leaves the problem of how to summarise the output meaningfully. However, I see that as a separate problem.

$\endgroup$
  • 1
    $\begingroup$ seems the link is broken $\endgroup$ – ed_sans May 24 at 2:24
  • $\begingroup$ I fixed the link by finding it on web.archive.org and providing link to a copy of thos slides hosted by the author on SlideShare. $\endgroup$ – Tim May 30 at 6:17
9
$\begingroup$

Gilles Celeux also worked on the problem of label switching, e.g.

G. Celeux, Bayesian inference for Mixture: the label switching problem. Proceedings Compstat 98, pp. 227-232, Physica-Verlag (1998).

As a complement to @darrenjw's fine answer, here are two online papers that reviewed alternative strategies:

  1. Jasra et al., Markov Chain Monte Carlo Methods and the Label Switching Problem in Bayesian Mixture Modelling
  2. Sperrin et al., Probabilistic relabelling strategies for the label switching problem in Bayesian mixture models
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy