1
$\begingroup$

I'm very confused, when I read on the wikipedia "From a mathematical standpoint, Rand index is related to the accuracy, but is applicable even when class labels are not used." Here and the formula of the Rand Index here

But when I use in R the rand.index function from fossil package and the Accuracy function from MLmetrics it doesn't give the same answer

> Accuracy(predicted, real) [1] 0.8266667

> rand.index(predicted,real) [1] 0.7114989

Please can someone explain to me the difference between these two and which one should I use?

NB:I use the k-medoids clustering algorithm (function pam from cluster package)

|cite|improve this question
$\endgroup$
1
$\begingroup$

Rand index is accuracy computed not in the raw data (which does not work unless you have you data where class 1 is cluster 1).

Instead, it is the accuracy on pairs of points, which is invariant to renaming clusters.

In binary classification, the common definition of accuracy is: (TP+TN)/(TP+FP+FN+TN), that should make the similarity of the equations ready to see.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thank you! can you develop a little bit more? what do you mean "where class 1 is cluster 1" $\endgroup$ – learneRS May 4 '18 at 11:08
  • 1
    $\begingroup$ What if the first class is 'healthy'? Which cluster is this? $\endgroup$ – Anony-Mousse May 4 '18 at 23:43
  • $\begingroup$ You mean if for example I have two classes: 'healthy' and 'sick' , the result of my clustering is 'cluster1' and 'cluster2'. The Rand index doesn't take into account if I name cluster1=healthy and cluster2=sick. Rand index would give the same result if I have chosen cluster1=sick and cluster2=healthy ? In contrast to the "accuracy"? $\endgroup$ – learneRS May 7 '18 at 8:31
  • 1
    $\begingroup$ Yes, the Rand index is invariant to permutation of clusters, and that is necessary. The Wikipedia description of it is pretty bad, as it does not define how to compute TP etc. for pairs. $\endgroup$ – Anony-Mousse May 7 '18 at 17:54
  • 1
    $\begingroup$ See en.m.wikipedia.org/wiki/Evaluation_of_binary_classifiers $\endgroup$ – Anony-Mousse May 9 '18 at 0:48
1
$\begingroup$

The confusion matrix that you use to calculate RI is different than that of accuracy. Definition of confusion matrix in the Rand Index (RI):

+--------------------------------+--------------------------------------+
| TP:                            | FN:                                  |
| Same class + same cluster      | Same class + different clusters      |
+--------------------------------+--------------------------------------+
| FP:                            | TN:                                  |
| different class + same cluster | different class + different clusters |
+--------------------------------+--------------------------------------+

Another difference between these two is that unlike accuracy, RI is mainly used for clustering (unsupervised learning).

The best link to learn RI is Introduction to Information Retrieval book: https://nlp.stanford.edu/IR-book/html/htmledition/evaluation-of-clustering-1.html

Accuracy is sensitive to cluster naming; however, RI is not.

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.