As I can read from the source code of scikit-learn, the stopping criteria for the iterative algorithm of Expectation Maximization (in my case applied to fitting Gaussian mixture models) is to put a threshold on the changes in the log-likelihood.

I can easily imagine that is way simpler to take some sample from the dataset and estimate the log-likelihood than putting a distance on covariance matrices and the centroids, to mimic what people do in k-means.

My question is: have you seen in any paper/software that proposed a threshold on the covariance matrices and the centroids of the Gaussian? Would this be silly to consider or can make sense sometimes?


  • 1
    $\begingroup$ In essence, it should be equivalent, as the log likelihood is continuous in the parameters (in most practical points). However, you can't use the exact same threshold. The scaling of the threshold could be dependent on the dimension (and maybe even the true parameters) $\endgroup$
    – tmrlvi
    Feb 5, 2019 at 23:43
  • $\begingroup$ It is data dependent. They need to have 3-4 special cases in there. Sometimes it gets caught in a cycle a -> b -> a -> b ... and can't get out. Some points try to get pathological, so if number of samples per element becomes 1 there can be a big problem. The intrinsic noise in the data only supports a minimum error, after that it is all false theory. $\endgroup$ Sep 30, 2020 at 12:32


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.