1
$\begingroup$

I'm primarily a programmer and have little to no training in formal maths or statistics of any kind.

I'm working on my dissertation (which foolishly is about clustering data), the process is unsupervised so the cluster labels are scattered and some are differently sized to the ground truth.

I've looked about and read about confusion matrices, the rand index and purity etc and I assume what I'm looking for is an external validation method but which one is a little bit beyond me.

I'm looking to calculate the accuracy of the cluster outputs, but some are differently sized to the ground truth and since its unsupervised the cluster labels won't match. What method of validation should I be using?

Improve this question
$\endgroup$
8
  • $\begingroup$ Check out the adjusted rand index. It's an external metric that can handle when the number of clusters doesn't match the number of groups in the "ground truth" labels. $\endgroup$
    – dmartin
    Commented Apr 12, 2016 at 16:23
  • $\begingroup$ This looks like exactly what I'm looking for. I'm struggling with the formula a little, but I've found a couple bits that should help out. Thank you. $\endgroup$
    – Malii
    Commented Apr 12, 2016 at 21:41
  • $\begingroup$ To use rand index or any of the external measures you do need labels, but your question title says you don't have labels. Your question body however appears as if you have labels... so what is it? do you want to compare two sets of labels, or not? $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Apr 13, 2016 at 7:16
  • $\begingroup$ Have a look at ELKI which includes several evaluation measures and many clustering algorithms. It's probably the best tool right now for clustering: quipu-strands.blogspot.de/2015/12/… $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Apr 13, 2016 at 7:18
  • $\begingroup$ Sorry @Anony-Mousse I meant the cluster labels are scattered, so the output from my programs the labels don't match the ground truth labels (eg: cluster 1 in the ground truth might contain all A's, but in the output cluster 2 contains the A's) and the code is already written for the clustering, I've got to write my own evaluation method (the whole thing has been very much re-inventing the wheel). Thank you though, I'll bare ELKI in mind for future use. $\endgroup$
    – Malii
    Commented Apr 13, 2016 at 10:13

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.