0
$\begingroup$

I am interested in quantifying how well a multi label assignment performs.

E.g. given 3 coloured boxes red, green and blue, with 20 likewise coloured balls in each.

A monkey is handed all the balls and puts them back in the boxes.

How can I quantify how well the monkey performed, with respect to getting the right balls in the right boxes?

$\endgroup$
  • $\begingroup$ A simple measure could be to sum the number of correctly placed balls and divide by the total number of balls. This however does not take into account if one box was solved perfect and the remaining ones equally bad? $\endgroup$ – LeonDK Mar 30 '16 at 11:06
0
$\begingroup$

The question "how can I" is the wrong approach. You could to this by taking the true-positives, but for some vague reason that is deemed unsatisfactory by you. You could also do this by just saying "The answer is 42". It's not very helpful, but it's a valid "how".

You need to be more precise on what you want to achieve, formally. "I don't think true-positive is what I want" is not formalmenough. What's wrong with using true positives? What's wrong with using thr confusion matrix? What other statistics can you apply to the confusion matrix? How about F1? Geometric mean of the TPR of each class?

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ If you were to ask a question within a given area, where it apparently was obvious that you were asking because you did not know the answer nor the appropriate considerations - Would you find your condescending answer useful? $\endgroup$ – LeonDK Apr 4 '16 at 8:19
  • $\begingroup$ Yes. Because it's you who needs to do these considerations; there is no "right" or "wrong" (42 can be "right"). Again: what is wrong with using precision? Apparently you don't want the usual solution; but then you need to know what you want different... $\endgroup$ – Has QUIT--Anony-Mousse Apr 4 '16 at 10:42
0
$\begingroup$

The most common evaluation metric for Multilabel tasks is, I believe, F1-score. There are two variants. Macro F1 is the average F1 score for all labels, while micro F1 score is the average F1 score for all instances predicted.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.