2
$\begingroup$

I have a data set of 54000 genes and I used different methods for clustering such as HAC, K-means, model based clustering and CLARA. The objective is to compare these methods. I used the Adjusted Rand Index. But there is something that I do not understand.

With my data set, the ARI value between a clustering result obtained by K-means and another clustering result, also obtained by K-means, with the same number of clusters (i.e I effectuated K-means two times) , is only 0.40, which is not a high value.

My question is, if the ARI value is not high for the same method compare to itself, can we use ARI to compare the clustering results for different method? And is there other index or method to compare them? I already read the topic How to select a clustering method? How to validate a cluster solution (to warrant the method choice)? but I still do not understand which methods are used to compare the clustering results.

$\endgroup$
11
  • 1
    $\begingroup$ In which respect did your two k-means analyses of the same dataset differ? $\endgroup$
    – ttnphns
    Jul 10, 2019 at 2:33
  • 1
    $\begingroup$ Are your data quantitative? K-means is appropriate mostly for quantitative data. $\endgroup$
    – ttnphns
    Jul 10, 2019 at 2:37
  • $\begingroup$ Thank you for your answer. My data is quantitative and in R, I used the command kmeans(data, centers = 9) to obtain the two k-means analyses. I just run at two different times. I also ran many times, and the ARI varies between 0.11 and 1. Can we use ARI for a method which is very sensitive to the initial choice of center like K-means? $\endgroup$
    – kaitokid
    Jul 10, 2019 at 4:44
  • 1
    $\begingroup$ "At different times" What do you mean? You mean you took the same data once then again? Was initial centers the same? Well, even if they aren't we should expect (with quantitative continuous data) quite similar results anyway. Perhaps you ought to tell more about your cluster analysis. $\endgroup$
    – ttnphns
    Jul 10, 2019 at 7:45
  • 1
    $\begingroup$ Are you sure there is any cluster structure in your data? And are you correct at your number of clusters? Did you check results first by some internal cluster validity indices (such as Calinski-Harabasz)? Also see if there are redundant dimensions (do PCA - maybe better to do clustering on few PCs than on 18 dimensions?). Before that is all checked there is no much sense in doing comparisons by external validity indices such as Rand or Adj. Rand. $\endgroup$
    – ttnphns
    Jul 10, 2019 at 10:40

2 Answers 2

1
$\begingroup$
  1. You are making a fallacy when saying if the ARI value is not high for the same method compare to itself, can we use ARI to compare the clustering results for different method. Cluster analysis results, most methods including K-means, are much dependent on its input "tuning" parameters (for K-means these are initial center seeds), and on data preprocessing. Your two runnings of K-means - which results you are comparing - differed, I suppose, in some of this respect (which, by the way? you haven't expressed it). Why do you expect the results must be very similar? They have not to. Especially if there is hardly any cluster structure in the data or the number of clusters was wrong. There is no reason to think, a priori and generally, that the difference in results between two runnings of the same method under different parameters ought to be less than of between two different methods.

  2. ARI's baseline (value 0) is not the absence of matching (similarity in results) but the level of chance matching. So value $0.40$ is not a low value, it is medium size value, I would say. But what is unadjusted Rand value, did you check? It will be higher.

  3. There are many "external clustering criteria" besides Rand or Adjusted Rand. See some of their formulae in the description of !cluagree SPSS macro of mine on my web page (download collection named "Compare partitions" there).

$\endgroup$
1
$\begingroup$

K-means is randomized.

Running it two times may result in quite different clusterings. In particular when it does not work well, it tends to produce very different results. On those few data sets where k-means works well, it usually produces similar results. It is perfectly in line with theory if k-means results are not similar to each other.

As mentioned in the other answer, 40% more than random (ARI, not Rand index. You may want to also report the Rand index itself) is not too bad. It probably means some of the k-means clusters agree, while others don't. That is to be expected that some do and some don't.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.