I have a dataset of items and I want to measure how my clustering method works. I'm using R and simple k-means clustering. I have a lets say gold standard of my clusters and I want to see how good is the result of my clustering algorithm based on the features I used. Is there any straightforward way to do that in R? I was thinking about something like Jaccard similarity per item. I mean the intersection/union of clusters containing each item and somehow mixing them together!

migrated from stackoverflow.com Aug 21 '14 at 10:14

This question came from our site for professional and enthusiast programmers.

  • This doesn't sound like a programming question. It sounds like you need advice on statistical methods for comparisons of clustering methods. This may be a better fit for Cross Validated. – MrFlick Aug 20 '14 at 22:48

Although the rand index is very popular, I suggest using either the Variation of Information (VI) metric or the split/join distance. Both are much better behaved, are proper distances (satisfy the triangle inequality) and are not (or not as much in the case of VI) affected by cluster sizes. See also e.g. Comparing clusterings: Rand Index vs Variation of Information and How to interpret these indices/metrics for comparing partitions intuitively out of these images?. My answer in the first of these goes into quite some depth; it is useful to take into account consistency, that is, clusterings can be quite different but still consistent in the sense that one is a subclustering of the other.

  • After reading the VI paper I believe it to be a better method. Thanks for the suggestion. – pbible Sep 3 '14 at 12:33

What you are looking for is the adjusted rand index in the mclust package. It measures the correlation between two partitions or clusterings. Check the wiki. You would compare it to your 'gold standard' to see how well they agree.

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.