Mclust: Data frame order affects solution

Question

I've come across some behavior in mclust::Mclust that I would not have expected, which is that the order of variables in the data passed to Mclust affects the solution it comes up with.

In the example below, the first ordering of the variables produces 2 profiles, and the second produces 6. However, calling either of these orders does produce a stable solution (e.g., the first call to Mclust will always produce 2 profiles).

> testData <- read.file(f = "http://fimi.ua.ac.be/data/chess.dat")

> summary(Mclust(subset(testData, select = c(X1, X3, X5, X7, X9, X11))))
----------------------------------------------------
Gaussian finite mixture model fitted by EM algorithm 
----------------------------------------------------

Mclust EEV (ellipsoidal, equal volume and shape) model with 2 components:

 log.likelihood    n df      BIC      ICL
       3547.068 3195 49 6698.738 6692.126

Clustering table:
   1    2 
2759  436 


> summary(Mclust(subset(testData, select = c(X11, X9, X1, X3, X5, X7))))
----------------------------------------------------
Gaussian finite mixture model fitted by EM algorithm 
----------------------------------------------------

Mclust EEV (ellipsoidal, equal volume and shape) model with 6 components:

 log.likelihood    n  df      BIC      ICL
       18473.94 3195 137 35842.37 35834.51

Clustering table:
  1   2   3   4   5   6 
431 932 210 881 524 217 

My suspicion is that the initialization step (that the documentation suggests is a hierarchical clustering procedure) is affected by the order of variable passed, but I'm not sure if there's a straightforward way to overcome this.

Update: It appears that there is something to my suspicion that the initialization step is involved in this behavior. If I load the same dataset as above, but instead pass Mclust an hcPairs list to its initialization argument that's based on an EII model, I get stable results across variable orders:

> testData <- read.file(f = "http://fimi.ua.ac.be/data/chess.dat")

> hc <- hc(subset(testData, select = c(X1, X3, X5, X7, X9, X11)), modelName = "EII")
> summary(Mclust(subset(testData, select = c(X1, X3, X5, X7, X9, X11)), initialization=list(hcPairs = hc)))
----------------------------------------------------
Gaussian finite mixture model fitted by EM algorithm 
----------------------------------------------------

Mclust VEV (ellipsoidal, equal shape) model with 2 components:

 log.likelihood    n df      BIC      ICL
       3391.936 3195 50 6380.405 6380.254

Clustering table:
   1    2 
1386 1809 
> summary(Mclust(subset(testData, select = c(X11, X9, X1, X3, X5, X7)), initialization=list(hcPairs = hc)))
----------------------------------------------------
Gaussian finite mixture model fitted by EM algorithm 
----------------------------------------------------

Mclust VEV (ellipsoidal, equal shape) model with 2 components:

 log.likelihood    n df      BIC      ICL
       3391.936 3195 50 6380.405 6380.254

Clustering table:
   1    2 
1386 1809 

Interestingly, this doesn't match either of the first two cluster/profile solutions above (the ns for the two-profile solution above don't match). I also don't have a clear understanding of why an EII model in the hc step produces these effects, I simply tried all possible models in the hc step and EII was the only one that produced these stable solutions.

Can anyone confirm that this is a sound way to go about this type of profile analysis, or offer an alternate solution?

Answer 1

You may try SVD transformation described in Scrucca and Raftery (2015). In this paper they pointed out the order of variables of your input data may affect your initial output of hc algorithm.

Mclust package does offer such options. Just like

mod <- Mclust(X, initialization = list(hcPairs = hc(X, use = “SVD”)))