2
$\begingroup$

I have a dataset of items and I want to measure how my clustering method works. I'm using R and simple k-means clustering. I have a lets say gold standard of my clusters and I want to see how good is the result of my clustering algorithm based on the features I used. Is there any straightforward way to do that in R? I was thinking about something like Jaccard similarity per item. I mean the intersection/union of clusters containing each item and somehow mixing them together!

$\endgroup$
  • $\begingroup$ This doesn't sound like a programming question. It sounds like you need advice on statistical methods for comparisons of clustering methods. This may be a better fit for Cross Validated. $\endgroup$ – MrFlick Aug 20 '14 at 22:48
2
$\begingroup$

Although the rand index is very popular, I suggest using either the Variation of Information (VI) metric or the split/join distance. Both are much better behaved, are proper distances (satisfy the triangle inequality) and are not (or not as much in the case of VI) affected by cluster sizes. See also e.g. Comparing clusterings: Rand Index vs Variation of Information and How to interpret these indices/metrics for comparing partitions intuitively out of these images?. My answer in the first of these goes into quite some depth; it is useful to take into account consistency, that is, clusterings can be quite different but still consistent in the sense that one is a subclustering of the other.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ After reading the VI paper I believe it to be a better method. Thanks for the suggestion. $\endgroup$ – pbible Sep 3 '14 at 12:33
0
$\begingroup$

What you are looking for is the adjusted rand index in the mclust package. It measures the correlation between two partitions or clusterings. Check the wiki. You would compare it to your 'gold standard' to see how well they agree.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy