4
$\begingroup$

It's typical to initialize EM for Gaussian Mixture Models using the result of kmeans clustering. However, kmeans only gives you the means (centers) of the starting GMM, but EM initialization often requires a complete GMM description (that is, including the covariance matrices and weights).

Therefore what is a 'good' way to come up with initial covariance matrices and weights for kmeans-based GMMs? Simply assign random values (assuming sum(weights)=1) ?

$\endgroup$
5
$\begingroup$

k-means also tells you which data points belong to which cluster. A good starting estimate for the covariances should be the within-cluster covariances, and a good estimate for the weights should be the fractions of data points allocated to each cluster.

$\endgroup$
0
$\begingroup$

Old question, but its worth saying anyway..

The EM algorithm iterates between finding the responsibilities for each data point (the probability they belong to each mixture) and calculating the modes of the distributions. Therefore you should only need to initialise the responsibilities OR the modes (but not both), this probably depends on the implementation.

K-means should give you the initialisation from the responsibilities, albeit with a probability of 1 or zero for each mixture.

That being said, A. Dondas answer is perfectly correct.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.