# Measure to compare quality of clustering results from different datasets

I have a correlation matrix that reflects the structural connectivity patterns of the human brain. I have this matrix for a population of subjects (73 subjects, so I have 73 correlation matrices originated from different datasets).

I applied a clustering algorithm to each matrix for a variety of number of clusters (from 2 to 5 clusters). Now, I want to choose the optimal number of clusters k. I'd like to find the same good k for all the 73 subjects, and therefore I want to do the choice based on the consistency across the population. So for each clustering solution (k=2,...,5), I want to measure its consistency for the entire population.

So, is there any statistical measure that I can use to compare the clustering results between the different datasets (subjects)?

(If anything is not clear, please let me know)

### Approach 1. Compare all the series of 292 cluster partitions directly for internal quality (see pt 3 here).

One way to compare cluster solutions is to apply some internal validity clustering criterion (which are plenty). We typically do that to select the "best" number of clusters. However, those same criterions can be used to choose the "best" partitioning method or the best-partitioned dataset of objects.

In respect to the latter, let me cite my document describing some such clustering criterions (see collection "clustering criterions" on my web-page"):

Usage: comparing not identical sets of objects. This is possible. One should understand that for a clustering criterion objects “i” in the set – are just anonymous rows. Therefore it will be correct to compare, by the criterion value, partitions P1 and P2 which partly or completely are comprised of not the same objects. At so doing k may be one or different in the partitions. However, if P1 and P2 consist of different number of objects one may use criterion only if it is insensitive to the number of objects N.

In your case you have 73 sets of objects - 73 correlation matrices. (You may just ignore that rows and columns in all the matrices represent the same brain parts: since individuals are different that is 73 different sets of some objects.) It is important and nice that the "calibre" of the 73 data sets is the same: it is correlation values, which are directly comparable across the sets. Also the size of the sets (matrices) is the same, so you can use any clustering criterion, not just "insensitive to N". As the citation goes, k (number of clusters) may be different.

So, select one or several clustering criterions to use (it could be, say, C-index or Dunn's; but given that Pearson r is convertable to the euclidean distance you could use virtually any criterion, any you find it worth). And compare the series of 73*4 (4 different k) = 292 partitions for their internal validity.

Observe the results (plot the criterion values, produce summary stats). What partitions have tended to be the "better"? Maybe most of the 73 3-cluster solutions prevail among the best? Then you can say k=3 should be chosen for all 73 subjects. Or maybe there are few subjects whose cluster results suggest different k? or no good k at all (subjects with no clear-cut cluster structure)?

### Approach 2. Cluster the 73 matrices by their similarity first.

Consider each subject (correlation matrix) as an object for clustering task. Since all the matrices are the same size, measure (correlation) and items (same brain parts) you may compute a similarity/distance measure between them as datasets and cluster them. Those proved to be quite similar so that might be seen as if "one subject" you might average into a single matrix. Consequently you end up with lesser number of matrices, subject types (say, 20) to cluster in each then. Best number of clusters k, sure, will need not be one the same for those 20 datasets, since they are knowingly quite different structures. That was a bit blunt, still potentially helpful approach.

• Thanks @ttnphns! I am trying the SPSS macros from your web-page. Just to check if I am doing everything right, I am loading the correlation matrix for one subject with the 4 clustering solutions. So, I have 206 variables (correlation matrix) and 4 variables that corresponds to the clustering solutions. Then I run the macro like this: !cindex matrix= v2 to v207 /clusol= clu2 clu3 clu4 clu5 /fast= YES  Is this okay? May 26 '18 at 11:46
• Ana, you may ask me by email (found on the web-page, author) about using a macro. May 26 '18 at 12:20
• For your comment. You will have to run a macro of a criterion 73 times - because you have 73 matrices. Each time you'll get 4 values of the criterion because 4 (k=2,3,4,5) cluster solutions were obtained on each data (matrix). You may then gether all the 73*4 criterion's values in one dataset to compare them as I've suggested in Approach 1. May 26 '18 at 12:24
• Btw, if your cluster analyses were based not on corr. but on abs(corr.) don't forget to abs(corr.) your matrices this time again. May 26 '18 at 12:27