1
$\begingroup$

I'm doing clustering via GMM, which is initialized first by k-means.

I am using a data matrix that cannot be classified as small by any standards, they are usually of the size 15000 x 1800, where 15000 are the number of observations and 1800 the size of each observation.

My reading of GMM in speaker recognition papers suggested that with large data we need higher number of gaussians, and most of the papers were using 512 or 1024 gaussians for best convergence.

For my data I tried with 128 and 256 Gaussians. I am using Matlab, but even after 300 iterations I am given a warning that the method failed to converge. But when I use a comparatively small value of k, such as 32. The methods converge without any problems and usually in less than 100 iterations.

I cannot explain this.

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ You have too few observations. A full GMM has a lot of parameters. With 1800 you should, as a rule of thumb, aim at having 3 times as many instances in each cluster (others will even recommend 3*d^2 instances)... $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Mar 18, 2016 at 6:51
  • $\begingroup$ @Anony-Mousse Thank you for your response. Can you please clarify this a bit more, as I do not fully understand what you mean by instances in each cluster and also what does d represent: as a rule of thumb, aim at having 3 times as many instances in each cluster (others will even recommend 3*d^2 instances). Many Thanks $\endgroup$
    – StuckInPhDNoMore
    Commented Mar 18, 2016 at 9:51
  • 2
    $\begingroup$ I used $d$ for dimensionality, since this is a vector space. Some people prefer to call this $p$. To get a reliable estimate of a covariance matrix (and thus, a reliable GMM), you need to have enough instances, $n\gg p$ in each cluster. Some people even say you should have $n>3p^2$ to get a sound result. For $n<p$ the covariance matrix becomes singular, see e.g. en.wikipedia.org/wiki/… $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Mar 18, 2016 at 12:35
  • $\begingroup$ @Anony-Mousse Thats great, thank you. Can you please add this as an answer so that I can accept it and close this question :) $\endgroup$
    – StuckInPhDNoMore
    Commented Mar 18, 2016 at 18:11

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.