5
$\begingroup$

I'm trying to apply some evaluation metrics to several clustering methods. I thought that I knew them basing on the multiclass confusion matrix, considering the rows as the actual classes and the columns as the predicted clusters:

  • TP would be the value in the main diagonal.
  • FN for each class would be the sum of all values in the corresponding row excluding (TP).
  • FP for each class would be the sum of all values in the corresponding column excluding the main diagonal element (TP).
  • TN for each class would be the sum of all the values of the confusion matrix excluding that class's row and column.

References:

However... when I'm trying to calculate the rand index, according to the book, Introduction to Information Retrieval and the example given by it, I found that they are considering the terms a, b, c and d of the rand index as TP, FP, FN and TN respectively. I don't find the relation between them if we take into account the actual definition of these terms and how to obtain the TP, FP, FN and TN in a confusion matrix, besides of they are considering all of them for all the classes.

Besides, then they calculate precision and recall based on these TP, FP, FN and TN... so it makes no sense for me, because it seems that they are giving a solution of an overall precision and recall regarding all the clusters.

Edit: It's not a problem about how calculate precision and recall for multiclass classification (I think that I have understood how to calculate them regarding my explanation and the first link that I have attached). I'm confused about the definitions given by the book (check the 2nd link) of TP, TN, FN and FP in the section "Evaluation of clustering" that I have attached previously, and all my questions are related to, precisely, the book.

Summing up, related to the book,

  • are these FP, TN, FN and TP definitions the same (the ones given by the book and the others ones that I have commented in the first part of my thread)?

  • are those precision and recall an overall solution of all the data clustering, instead giving the one for each class (I don't know if it makes sense an overall solution about these metrics)?

  • is accuracy = rand index?

There are 3 questions, but they are all related to the topic.

P. S. : All the edited parts are in italic, and the most important part and questions are in bold. I hope it will shed some light on the doubts about my questions

$\endgroup$
  • $\begingroup$ Possible duplicate of How to calculate precision and recall for multiclass classification using confusion matrix? $\endgroup$ – Tim Mar 18 '16 at 11:01
  • 2
    $\begingroup$ @Tim No, it's not a duplicate. That thread you have attached it's only about how calculate precision and recall for multiclass classification, and I haven't that problem (in fact, I would have liked to attach that thread as a reference, but I couldn't because of my reputation, and I think that I have understood how to calculate them regarding my explanation). I'm confused about the definitions given by the book (check the 2nd link) of TP, TN, FN and FP in the section "Evaluation of clustering" that I have attached previously, and all my questions are related to, precisely, the book. $\endgroup$ – Emilio Genaro López Mar 18 '16 at 12:27
  • 3
    $\begingroup$ This may be distinctive enough to stay open IMO. $\endgroup$ – gung Mar 18 '16 at 12:45
  • 2
    $\begingroup$ These are all artificial constructs and do not allow for "close calls". I suggest you apply proper probability accuracy scoring rules to the matrix of predicted probabilities. $\endgroup$ – Frank Harrell Mar 18 '16 at 13:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.