I know that a similar subject was treated here, but my question is a little bit different.
I have a result of multilabel classification, like this (2 observations, 3 labels in the example, in practice I have 10k observations and 300 labels):
> pred_df truth.label1 truth.label2 truth.label3 pred.label1 pred.label2 pred.label3 1 TRUE FALSE FALSE TRUE TRUE FALSE 2 FALSE FALSE TRUE FALSE FALSE TRUE
I know that confusion matrix deals with accurracy of class/labels prediction, but I was wondering if it still has a meaning if applied to the observations instead. Indeed, I have the idea to transpose my results and compute the confusion matrix for each observation:
> t(pred_df) 1 2 truth.label1 TRUE FALSE truth.label2 FALSE FALSE truth.label3 FALSE TRUE pred.label1 TRUE FALSE pred.label2 TRUE FALSE pred.label3 FALSE TRUE #confusion matrix for observation 1 : cm1 <- confusionMatrix(t(pred_df)[1:3,1],t(pred_df)[4:6,1]) #confusion matrix for observation 2 : cm2 <- confusionMatrix(t(pred_df)[1:3,2],t(pred_df)[4:6,2])
It seems to me that this will measure the accuracy of my model for each observation, then I could summarize all the confusion matrices to have a good metric for the whole multilabel classification... But I am not sure it still has a relevant signification (practically and theoretically speaking). Does it?