I'm primarily a programmer and have little to no training in formal maths or statistics of any kind.

I'm working on my dissertation (which foolishly is about clustering data), the process is unsupervised so the cluster labels are scattered and some are differently sized to the ground truth.

I've looked about and read about confusion matrices, the rand index and purity etc and I assume what I'm looking for is an external validation method but which one is a little bit beyond me.

I'm looking to calculate the accuracy of the cluster outputs, but some are differently sized to the ground truth and since its unsupervised the cluster labels won't match. What method of validation should I be using?

  • $\begingroup$ Check out the adjusted rand index. It's an external metric that can handle when the number of clusters doesn't match the number of groups in the "ground truth" labels. $\endgroup$ – dmartin Apr 12 '16 at 16:23
  • $\begingroup$ This looks like exactly what I'm looking for. I'm struggling with the formula a little, but I've found a couple bits that should help out. Thank you. $\endgroup$ – Malii Apr 12 '16 at 21:41
  • $\begingroup$ To use rand index or any of the external measures you do need labels, but your question title says you don't have labels. Your question body however appears as if you have labels... so what is it? do you want to compare two sets of labels, or not? $\endgroup$ – Has QUIT--Anony-Mousse Apr 13 '16 at 7:16
  • $\begingroup$ Have a look at ELKI which includes several evaluation measures and many clustering algorithms. It's probably the best tool right now for clustering: quipu-strands.blogspot.de/2015/12/… $\endgroup$ – Has QUIT--Anony-Mousse Apr 13 '16 at 7:18
  • $\begingroup$ Sorry @Anony-Mousse I meant the cluster labels are scattered, so the output from my programs the labels don't match the ground truth labels (eg: cluster 1 in the ground truth might contain all A's, but in the output cluster 2 contains the A's) and the code is already written for the clustering, I've got to write my own evaluation method (the whole thing has been very much re-inventing the wheel). Thank you though, I'll bare ELKI in mind for future use. $\endgroup$ – Malii Apr 13 '16 at 10:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.