3
$\begingroup$

I am doing an unsupervised clustering analysis for a genomics project. This means that I do not know when a particular clustering analysis is good or not.

I am running different clustering algorithms and different 'sets of features'. What I mean with different 'sets of features' is that given a data frame, I choose different combination of columns depending on its biological importance. For instance, some variables measure things at the sequence level, while others are measuring a particular cellular process or some other feature that cannot be measured at the sequence level. I am playing around with the different outputs of these sets of features, running the algorithms with all the features, or ignoring some, etc .

What I want is to compare the different clusters of these different runs and see if some of my objects are being clustered similarly despite lacking some sets of features. Does this make sense?

Is there any recommendation on how can I do this?

$\endgroup$
5
$\begingroup$

You can use the Adjusted Rand Index or the Adjusted Mutual Information to measure the similarity (agreement) of the overall results of two clustering algorithms on an overlapping dataset.

Both scores are adjusted for chance which means that 2 random clusterings will likely have an ARI or AMI close to 0.0.

Furthermore you can use those measure for model selection (e.g. finding the number of k in k-means) by running the clustering algorithm twice on 2 overlapping samples of the datasets and measuring the agreement on the overlap. The assumption is that a high agreement on the overlap means a higher stability of the algorithm and hence a better value for k (it captures better the real structure of the dataset).

A Novel Approach for Automatic Number of Clusters Detection in Microarray Data based on Consensus Clustering by Nguyen and Epps is probably the best reference for this method and it is further applied to microarray data.

$\endgroup$
  • $\begingroup$ I looked on the formula for computing the ARI and AMI measures, but it's a little confusing. Say I want to compare 4 clustering algorithms using this measures, can I compute them for each clustering algorithm separately, or I need to get the representatives produced all this 4 algorithm, and compare them two by two ? Given a test dataset and a clustering algorithm that produced a set of K representatives, is it possible to compute the ARI and AMI measures for this algorithm ? Can you provide a simple pseudo-code to compute ARI and AMI measures, because the formulas are a little confusing. Thx. $\endgroup$ – shn Mar 9 '12 at 8:52
  • 2
    $\begingroup$ ARI and AMI give you agreement between 2 clusterings. If you have some ground truth you can use it as a reference to compute absolute scores against it. Otherwise you will have to can just consider pairwise agreements. As for implementations have a look at: github.com/scikit-learn/scikit-learn/blob/master/sklearn/… $\endgroup$ – ogrisel Mar 10 '12 at 3:33
  • $\begingroup$ @ogrisel: The link doesn't work $\endgroup$ – Diego Oct 8 '12 at 19:16
  • $\begingroup$ I just checked and it seems to work for me. The documentation is here: scikit-learn.org/stable/modules/… $\endgroup$ – ogrisel Oct 9 '12 at 5:16
  • $\begingroup$ @ogrisel sorry I didn't specify the broken link: Nguyen $\endgroup$ – Diego Oct 10 '12 at 19:11
3
$\begingroup$

I only complete two mentioned measures with third that is also widely used in this situation, Normalized Variation of Information, which was proposed in the last 10 years. It is also scaled to the unit interval.

In addition, I recommend that you use any ensemble clustering technique that can assign to more partitions, so called consensus partitions, where objects are often better partitioned than in the initial partitions.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.