0
$\begingroup$

I have the following data, which when plotted as a histogram, are a mixture of Gaussians:

enter image description here

I would like to write an algorithm that would infer:

(1) the number of "peaks" or normal distributions in this data (i.e. three in the above) (2) the mean/position of each gaussian (3) the variance of each Gaussian (4) the height of each Gaussian/peak

However, I do not know a prior the number of Gaussians in the dataset. Sometimes, there could be four:

or maybe even seven:

enter image description here

If this is an application of the EM algorithm, the latent variables are mean \mu, variance \sigma, and the parameter is k, the number of peaks? As I don't know the number of clusters a priori, is this a different problem?

It seems like I'm using density estimation/unsupervised learning first to infer the number of clusters/gaussians, and then EM to infer the parameters (i.e. the information of each peak). What solutions exist for this problem?

Improve this question
$\endgroup$
  • $\begingroup$ It's "easy" when there is a relatively clean separation between the individual Gaussians, as in your plots. When there is significant overlap, things get trickier. $\endgroup$ – Mark L. Stone Aug 8 '16 at 23:42
  • $\begingroup$ @MarkL.Stone For the data I'm working with, there is no overlap like you might imagine. $\endgroup$ – ShanZhengYang Aug 8 '16 at 23:46
  • 1
    $\begingroup$ The title is somewhat misleading as what is given seems to be data (iid? samples from the distribution?) 'Given a pdf' I would understand to mean that the 'true' density function is known in some sense (e.g. we can evaluate it pointwise). $\endgroup$ – Juho Kokkala Aug 9 '16 at 4:50
  • 1
    $\begingroup$ There are EM-like algorithms that do not need to know the exact number of components a priori. For example the variational Gaussian Mixture Model as described by Bishop in his book "Pattern Recognition and Machine Learning". Another example are Dirichlet Process GMMs, there is an implementation of the latter in scikit-learn for python (scikit-learn.org/stable/modules/mixture.html) (their VGMM implementation does not reduce component count by itself) $\endgroup$ – nyro_0 Aug 12 '16 at 9:29
  • 1
    $\begingroup$ There is no reason for the peaks to connect with the number of components: take the instance of a mixture of Gaussians with the same mean but different variances. $\endgroup$ – Xi'an Nov 8 '16 at 17:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.