2
$\begingroup$

I am attempting to cluster data using Mclust. The data is originally from a dissimilarity matrix, transformed via multidimensional scaling in R (MASS::isoMDS). As I experiment with different numbers of dimensions, I have found that with larger numbers of dimensions (~50 relative to ~10 for the smaller values tried) certain model types return NA values for the BIC with larger numbers of components, i.e. clusters.

The Mclust manual states

The missing values correspond to models and numbers of clusters for which parameter values could not be fit (using the default initialization). For multivariate data, the default initialization for all models uses the classification from hierarchical clustering based on an unconstrained model.

Can anyone explain what causes the failure of the parameter values to fit, and how that relates to the number of dimensions being used? Knowing this might help me better understand the potential value or lack thereof of those later dimensions.

In case it is relevant, in the cases I have tried the model type VVV stops at a lower number of components followed by EVI and VEV, then by VEI and VII, with the other 4 (EEI, EII, EEE, and EEV) returning values up to the default max of 9 components. I can run any number of other numbers of dimensions to derive a clearer pattern if that is helpful.

$\endgroup$
  • $\begingroup$ Clusters can disappear. And since the covariance matrix has $O(d^2)$ entries, you might be overfitting if you don't have a whole lot of observations (and then, MDS and Mclust will likely take a long time to run). $\endgroup$ – Has QUIT--Anony-Mousse Sep 25 '13 at 17:45
  • $\begingroup$ The MDS can certainly take some time to run, especially with my larger data set, but what I ran on has about 900 data points, which goes quickly enough and should be plenty for the task. More importantly in this particular case, one of the failing models is VII, which doesn't involve a covariance matrix since the only things varying about the clusters is their proportion (density) and size (standard deviation). $\endgroup$ – Todd Gillette Sep 25 '13 at 18:24
  • 1
    $\begingroup$ 900 sounds way too small for full covariance matrixes. See: 45^2 entries per matrix, and now assume you have just 2 clusters, that is 4140 parameters in your model. As a rule of thumb, you should have $3*d^2$ observations in each cluster at minimum. $\endgroup$ – Has QUIT--Anony-Mousse Sep 25 '13 at 20:22
  • $\begingroup$ Even with simpler models, that is a good rule of thumb, $3*d^2$ per cluster. $\endgroup$ – Has QUIT--Anony-Mousse Sep 25 '13 at 20:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.