0
$\begingroup$

When generating Gaussian mixture models using expectation maximization with Bayesian Information Criterion, is it necessary to report a confidence measure? How do you know that the algorithms are returning the optimal models?

I'm playing around with densityMclust in the mclust R package, and it doesn't seem to be returning any confidence measure (analogous to a p-value).

$\endgroup$
1
$\begingroup$

I will start answering this questions in the reverse order, as it seems to make more sense.

I'm playing around with densityMclust in the mclust R package, and it doesn't seem to be returning any confidence measure (analogous to a p-value).

It seems to me that R package mclust used to have confidence measures reporting functionality in some of its previous versions, but it has been removed or disabled for some reasons. That functionality included calculating (via bootstrapping) and reporting significance (p-values) as well as standard errors and confidence intervals for estimated parameters. Based on current CRAN documentation, the functionality was available via functions mclustBootstrapLRT() and MclustBootstrap().

Considering the above, I think that you have the following options:

  1. Determine the latest version of mclust, which contained needed functionality, install that version and perform the analysis.

  2. Implement missing functionality in end-user R code, based on information, formulas and references, provided in the documentation's description for mclustBootstrapLRT() and MclustBootstrap() functions. IMHO, a much better source of information for manual implementation is a nice blog post " EM Algorithm: Confidence Intervals" by Stephanie Hicks.

  3. Consider using mixtools package, which seems to contain at least significance (p-values) calculating and reporting functionality, similar to the one of mclustBootstrapLRT() function (see page 26 in the corresponding JSS paper).

When generating Gaussian mixture models using expectation maximization with Bayesian Information Criterion, is it necessary to report a confidence measure?

Unless it is very difficult (skill-wise or time-wise) for you to use one of the above-mentioned options, I think that it is quite important to include such reporting in your analysis' results, as it demonstrates (academic or industrial) professional level of statistical rigor.

How do you know that the algorithms are returning the optimal models?

I think that EM algorithm returns optimal models, because the M-step is the optimizing one (M from maximization). Having said that, EM algorithm iterates until it converges to a local maximum of the log-likelihood function.

Additional information on EM algorithm can be found in the following papers: brief, medium and large (a 280+ pages book, ironically called "gentle tutorial" :-). It might also be of interest this paper on estimating standard errors for EM algorithm and this general paper on estimating confidence intervals for mixture models.

$\endgroup$
3
  • $\begingroup$ I don't have time now to provide a properly formatted list of references - plan to do that eventually, though. $\endgroup$ – Aleksandr Blekh Apr 16 '15 at 3:26
  • $\begingroup$ Thanks for your comment Aleksandr! I have tried experimenting with mixtools, but found some problems with the their mixture modeling functionality. I'm hoping to implement these confidence intervals soon, but knowing that these models are indeed the optimal model is suitable for now. What is the worst (?) confidence you've personally seen on such an optimal model? $\endgroup$ – swetharevanur Apr 24 '15 at 4:10
  • $\begingroup$ Also, I was wondering if I could get your feedback on this other question I've asked on CrossValidated: LRT where probabilities of being in components are same? $\endgroup$ – swetharevanur Apr 24 '15 at 4:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.