5
$\begingroup$

I'm very confused, when I read on the wikipedia "From a mathematical standpoint, Rand index is related to the accuracy, but is applicable even when class labels are not used." Here and the formula of the Rand Index here

But when I use in R the rand.index function from fossil package and the Accuracy function from MLmetrics it doesn't give the same answer

> Accuracy(predicted, real) [1] 0.8266667

> rand.index(predicted,real) [1] 0.7114989

Please can someone explain to me the difference between these two and which one should I use?

NB:I use the k-medoids clustering algorithm (function pam from cluster package)

Improve this question
$\endgroup$
4
$\begingroup$

Rand index is accuracy computed not in the raw data (which does not work unless you have you data where class 1 is cluster 1).

Instead, it is the accuracy on pairs of points, which is invariant to renaming clusters.

In binary classification, the common definition of accuracy is: (TP+TN)/(TP+FP+FN+TN), that should make the similarity of the equations ready to see.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thank you! can you develop a little bit more? what do you mean "where class 1 is cluster 1" $\endgroup$
    – learneRS
    May 4 '18 at 11:08
  • 1
    $\begingroup$ What if the first class is 'healthy'? Which cluster is this? $\endgroup$
    – Has QUIT--Anony-Mousse
    May 4 '18 at 23:43
  • $\begingroup$ You mean if for example I have two classes: 'healthy' and 'sick' , the result of my clustering is 'cluster1' and 'cluster2'. The Rand index doesn't take into account if I name cluster1=healthy and cluster2=sick. Rand index would give the same result if I have chosen cluster1=sick and cluster2=healthy ? In contrast to the "accuracy"? $\endgroup$
    – learneRS
    May 7 '18 at 8:31
  • 2
    $\begingroup$ Yes, the Rand index is invariant to permutation of clusters, and that is necessary. The Wikipedia description of it is pretty bad, as it does not define how to compute TP etc. for pairs. $\endgroup$
    – Has QUIT--Anony-Mousse
    May 7 '18 at 17:54
  • 1
    $\begingroup$ See en.m.wikipedia.org/wiki/Evaluation_of_binary_classifiers $\endgroup$
    – Has QUIT--Anony-Mousse
    May 9 '18 at 0:48
3
$\begingroup$

The confusion matrix that you use to calculate RI is different than that of accuracy. Definition of confusion matrix in the Rand Index (RI):

+--------------------------------+--------------------------------------+
| TP:                            | FN:                                  |
| Same class + same cluster      | Same class + different clusters      |
+--------------------------------+--------------------------------------+
| FP:                            | TN:                                  |
| different class + same cluster | different class + different clusters |
+--------------------------------+--------------------------------------+

Another difference between these two is that unlike accuracy, RI is mainly used for clustering (unsupervised learning).

The best link to learn RI is Introduction to Information Retrieval book: https://nlp.stanford.edu/IR-book/html/htmledition/evaluation-of-clustering-1.html

Accuracy is sensitive to cluster naming; however, RI is not.

Improve this answer
$\endgroup$
2
$\begingroup$

Couple of points to remember:

  1. Rand Index looks at similarity between any two clustering methods. Generally, there are no 'true' labels present here. Whereas for calculating accuracy, you need to compare the true labels with the predicted labels.
  2. Like Has QUIT--Anony-Mousse answered, Rand Index looks for the relationship between two points in the dataset, rather than the relationship of a point and its true label.

Here's some visualization:

  • If Method 1 and Method 2 are two clustering methods, then their Rand Index will be equal to 1.
  • If Method 2 signifies the Reference Clustering (where the two colors depict different classes), then accuracy will be 0.

Hope this helps.

enter image description here

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.