1
$\begingroup$

As part of my thesis, we wanted to investigate the ability of using z-scores as indicators for the homogeneity of clusters (or speaking more generally of partitions of a vector). After some modelling I believe this is not entirely possible and I would like to get some opinions on this.

Suppose I have some 2D data with one predictor c and my cluster assignment cl, which splits c into 5 groups of exactly 20 elements. There is little variation among the elements of c in every cluster and every cluster contains a distinct set of elements of c(i.e. cluster 1 contains mostly ones, cluster 2 mostly twos, a.s.o.). I then want to compute each indivdual cluster's z-score based on c as follows:

# library(data.table)
# dat<-as.data.table(dat)
meanCl<-dat[,mean(c),by=cl][,V1,]
mean0<-sapply(1:1000,function(x){
              mean(sample(dat[,c,],20,replace=T))}
sd0<-sd(mean0)
mean0<-mean(mean0)
z<-(meanCl-mean0)/sd0

... i.e. I compute the mean of c for every subgroup in cl. I go on to compute the mean of 1000 clusters containing 20 random elements from the distribution of c. I then calculate every clusters' z-score by subtracting its mean from the mean of the random clusters' means and divide by the standard deviation in those means to get:

[1] -4.96401933 -4.05057547 -0.03142247  2.89159789  5.99730703

So even though all clusters are distinctly characterised in their internal distribution of c, the z-score for cluster 3 is basically 0, as its mean tends towards the mean of the sample distribution of c.

Typing this out, it makes sense to me to not be able to use a cluster's z-score as a predictor for its homogeneity, however, as intially stated, I would like some ideas on this.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Are you trying to use z-score to describe homogeneity or uniqueness? You use different words in the title and in your closing paragraph. $\endgroup$
    – user77876
    Commented Aug 25, 2017 at 10:58
  • $\begingroup$ @user77876 Yes, homogeneity is what I meant to write in the closing paragraph. I edited the wording accordingly. $\endgroup$
    – victor_v
    Commented Aug 25, 2017 at 12:52
  • 1
    $\begingroup$ The description of the theoretical process being used to create these 1,000 randomly chosen clusters is unclear. You should define "homogeneity" more explicitly and benchmark results against standard metrics. Are the draws of 20 elements from the same population or data, i.e., are these bootstrapped draws? Why only 1,000 draws? Why not 1,000,000? Why is there only 1 predictor? Are five clusters always the result? Why five? A little more information would help. That said, if the elements in the clusters can be identified or tagged, why not track their diffusion in assignment across the draws? $\endgroup$
    – user78229
    Commented Aug 25, 2017 at 13:12
  • $\begingroup$ @DJohnson You're correct, I was a little vague. I refer to homogeneity of a cluster as the number and quantitiy of different elements in a cluster relative to the number and quantity of elements in the base distribution. Some other metrics I can think of would be e.g. shannon's entropy. The random draws are indeed bootstrapped (I included replace=T in the code). 1,000 random draws were based on this publication pnas.org/content/102/43/15545.full. 5 clusters and 1 predictor was an arbitrary choice. Measuring diffusion is an interesting idea! Can you recommend a good reference? $\endgroup$
    – victor_v
    Commented Sep 6, 2017 at 13:45
  • 2
    $\begingroup$ Yeah. It sounds like your underlying assumptions are information theoretic. One possible metric can be found in Marina Meila, Comparing Clusterings, University of Washington Statistics Technical Report 418 and COLT 03 paper...ungated copy here ... stat.washington.edu/mmp/Papers/compare-colt.pdf. Another approach could be Andreas Brandmaier's complexity-based permutation distribution clustering for time series. He has several ungated papers out there and an R module, e.g., here ... jstatsoft.org/article/view/v067i05/v067i05.pdf $\endgroup$
    – user78229
    Commented Sep 6, 2017 at 16:01

1 Answer 1

Reset to default
2
$\begingroup$

Your intuition is correct: the z-score of a cluster's mean with respect to the means of the other clusters cannot be used to determine its homogeneity. The reason for this is twofold.

  1. A cluster's homogeneity does not depend on other clusters. A cluster's homogeneity does not change depending on the location of the other clusters in the clustering; and

  2. A cluster's mean does not capture the "spread" of the instances assigned to it. The clusters $(3, 3, 3, 3, 3)$ and $(-1, 0, 2, 5, 9)$ both have a mean value of 3 but have ranges, for example, of $0$ and $10$.

Thinking specifically about a cluster's homogeneity I would suggest that you consider internal measures of cluster quality. These measure how well a given clustering groups similar instances together while keeping different instances apart. Internal measures such as the Davies-Bouldin index, Dunn index and Silhouette coefficient all explicitly consider the "spread" of instances assigned to a cluster.

Davies-Bouldin index: $DB = \frac{1}{n} \sum_{i=1}^n \max{j \neq i} \text{ } (\frac{\sigma_i+\sigma_j}{d(c_i,_j)})$

With $n$ the number of clusters, $c_x$ the centroid of cluster $x$, $\sigma_{x}$ is the average distance of all elements in cluster $x$ to centroid $c_{x}$, and $d(c_{i},c_{j})$ is the distance between centroids $c_i$ and $c_j$.

Dunn Index: $D =\frac{\min{i,j} d(i,j)}{\max{k} d'{k}}$

With $d(i,j)$ the distance between clusters $i$ and $j$, and $d'(k)$ the intra-cluster distance of cluster $k$.

Silhouette coefficient for instance $i$: $s(i) = \frac{b(i)-a(i)}{\max{(a(i),b(i))}}$

With $a(i)$ the average dissimilarity of $i$ with all other data within the same cluster, and $b(i)$ the lowest average dissimilarity of $i$ to any other cluster, of which $i$ is not a member. The silhouette coefficient of the clustering is the sum of $s(i)$ for all $n$ instances clustered.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for your comment. I was aware of the silhouette coefficient and some other metrics and will revert to those instead. I had just come across this idea and wanted to give it a try. $\endgroup$
    – victor_v
    Commented Sep 6, 2017 at 13:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.