19
$\begingroup$

Label switching (i.e., the posterior distribution is invariant to switching component labels) is a problematic issue when using MCMC to estimate mixture models.

  1. Is there a standard (as in widely accepted) methodology to deal with the issue?

  2. If there is no standard approach then what are the pros and cons of the leading approaches to solve the label switching problem?

Improve this question
$\endgroup$
2
  • $\begingroup$ I was considering asking "How can I do an MCMC model of the output on lmer for models with random slopes?" but I'm wondering whether that question is redundant with this one. That is, is the "label switching issue" when using MCMC to estimate mixture models the same sort of problem take makes it so that pvals.fnc() in languageR is able to MCMC intercept models but not models with slopes? If not, please let me know and I'll go back to asking my initial question. $\endgroup$
    – russellpierce
    Commented Dec 13, 2010 at 1:14
  • $\begingroup$ @drknexus I do not know R to comment in your question. Perhaps, you should just post your question with a comment that your qn may be lined to this one. $\endgroup$
    – user28
    Commented Dec 14, 2010 at 4:10

3 Answers 3

Reset to default
15
$\begingroup$

There is a nice and reasonably recent discussion of this problem here:

Christian P. Robert Multimodality and label switching: a discussion. Workshop on mixtures, ICMS March 3, 2010.

Essentially, there are several standard strategies, and each has pros and cons. The most obvious thing to do is to formulate the prior in such a way as to ensure there is only one posterior mode (eg. order the means of the mixuture components), but this turns out to have a strange effect on the posterior, and therefore isn't generally used. Next is to ignore the problem during sampling, and then post-process the output to re-label the components to keep the labels consistent. This is easy to implement and seems to work OK. The more sophisticated approaches re-label on-line, either by keeping a single mode, or deliberately randomly permuting the labels to ensure mixing over multiple modes. I quite like the latter approach, but this still leaves the problem of how to summarise the output meaningfully. However, I see that as a separate problem.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ seems the link is broken $\endgroup$
    – Edgar Santos
    Commented May 24, 2019 at 2:24
  • $\begingroup$ I fixed the link by finding it on web.archive.org and providing link to a copy of thos slides hosted by the author on SlideShare. $\endgroup$
    – Tim
    Commented May 30, 2019 at 6:17
10
$\begingroup$

Gilles Celeux also worked on the problem of label switching, e.g.

G. Celeux, Bayesian inference for Mixture: the label switching problem. Proceedings Compstat 98, pp. 227-232, Physica-Verlag (1998).

As a complement to @darrenjw's fine answer, here are two online papers that reviewed alternative strategies:

  1. Jasra et al., Markov Chain Monte Carlo Methods and the Label Switching Problem in Bayesian Mixture Modelling
  2. Sperrin et al., Probabilistic relabelling strategies for the label switching problem in Bayesian mixture models
Improve this answer
$\endgroup$
0
$\begingroup$

With the R-Package "label.switching" (https://cran.r-project.org/web/packages/label.switching/index.html), the following algorithms can be compared:

ECR algorithm (default version), ECR algorithm (two iterative versions), PRA algorithm, Stephens’ algorithm, Artificial Identifiability Constraint (AIC), Data-Based relabelling and a probabilistic relabelling algorithm (SJW).

With the function permute.mcmc() the labels in the mcmc chains can be relablled for parameters that differ between mixture components or latent classes.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.