2
$\begingroup$

I have some code that looks for clusters in x,y data. To check the number of clusters I use, I want to get the BIC. This is not possible (easily) using kmeans(), and so I've switched to the mclust package. Specifically, I'm trying to replace kmeans() from the R stats package, with Mclust() from the mclust package.

Using Mclust() requires me to specify which model should be used for the clustering. According to ?Mclust, the following models can be used in Mclust():

univariate mixture      
"E"  =   equal variance (one-dimensional)
"V"  =   variable variance (one-dimensional)
multivariate mixture        
"EII"    =   spherical, equal volume
"VII"    =   spherical, unequal volume
"EEI"    =   diagonal, equal volume and shape
"VEI"    =   diagonal, varying volume, equal shape
"EVI"    =   diagonal, equal volume, varying shape
"VVI"    =   diagonal, varying volume and shape
"EEE"    =   ellipsoidal, equal volume, shape, and orientation
"EEV"    =   ellipsoidal, equal volume and equal shape
"VEV"    =   ellipsoidal, equal shape
"VVV"    =   ellipsoidal, varying volume, shape, and orientation
single component        
"X"  =   univariate normal
"XII"    =   spherical multivariate normal
"XXI"    =   diagonal multivariate normal
"XXX"    =   ellipsoidal multivariate normal

I'm presuming that k-means in stats is a "spherical, unequal volume" model, ie. to get k-means(x = data, centers = 6) to match mclust(), I should use mclust(data, G = 6, modelNames = c("VII")).

However, in the limited tests I've done, this gives different cluster centroids. The example below uses 6 clusters with some test data. The centroids obtained through each method are shown.

enter image description here

Can anyone confirm which mclust() model is equivalent to kmeans()?

$\endgroup$
  • $\begingroup$ I doubt that there can be any equivalence. mclust presents normal-model-based EM clustering, wherease k-means is not dependent on any type of the distribution, it is not model-based. These are different algorithms. $\endgroup$ – ttnphns Aug 28 '14 at 6:45
  • $\begingroup$ I hadn't realised/noticed that Mclust() uses gaussian mixture modeling. Therefore there's no way the two methods are really comparable. I'll leave this question here for people that make the same mistake... $\endgroup$ – Andy Clifton Aug 28 '14 at 15:31
6
$\begingroup$

Gaussian Mixture Modeling is not the same as k-means.

None of the models has a 1:1 correspondence to k-means. The closes is probably

"EII"    =   spherical, equal volume

but Mclust will still use a soft assignment, whereas k-means uses a hard assignment. There is a closer relationship between EII and fuzzy c-means (although I don't think they are the same either, due to different weighting functions).

Also note that k-means is commonly randomly seeded, so different runs of k-means will often yield different results, too!

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I completely missed the bit about Mclust() using GMM. If I'd seen that (or not been so infatuated with the fact that it reported BIC), I'd never have posted this question... $\endgroup$ – Andy Clifton Aug 28 '14 at 15:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.