# Optimal number of components in a Gaussian mixture

So, getting an "idea" of the optimal number of clusters in k-means is well documented. I found an article on doing this in gaussian mixtures, but not sure I am convinced by it, don't understand it very well. Is there a ... gentler way of doing this?

• Could you cite the article, or at least outline the methodology it proposes? It's hard to come up with a "gentler" way of doing this if we don't know the baseline :) – jbowman Jul 16 '12 at 18:30
• Geoff McLachlan and others have written books on mixture distributions. I am sure these include approaches to determining the number of components in a mixture. You could probably look there. I agree with jbowman that relieving your confusion would best be achieved if you would indicate to us what it is that you are confused about. – Michael R. Chernick Jul 16 '12 at 19:05
• The Estimating Optimal Number of Gaussian Mixtures Based on Incremental k-means for Speaker Identification.... Is its title, it's free to download. It basically increments the number of clusters by 1 until you see that two clusters become dependant between each other, something like that. Thank you! – JEquihua Jul 18 '12 at 20:18
• Why not just choose the number of components that maximises the cross-validation estimate of the likelihood? It is computationally expensive, but cross-validation is hard to beat in most cases for model selection, unless there are a large number of parameters to tune. – Dikran Marsupial Dec 18 '13 at 15:58
• Can you explain a bit what the cross-validation estimate of the likelihood is? I'm not aware of the concept. Thank you. – JEquihua Dec 18 '13 at 19:36

The likelihood for GMM is just $p(x|\pi,\mu,\Sigma)=\sum_K\pi_kN(x|\mu_k,\Sigma_k)$ by definition, where $K$ is the number of components (clusters) and $\pi$,$\mu$,$\Sigma$ are model parameters. By changing the value of $K$ you can plot the GMM likelihood for training and validation sets like the following. Another commonly used the method is the Bayesian information criterion (BIC): $$BIC = -2\log(L)+K\log(n)$$ where $L$ is the likelihood, K the number of parameters and $n$ the number of data points. It can be understood as adding a penalty for the number of parameters to the log likelihood.