For a single label classification, the notion of confidence is easy to understand. If the classifier has 80% confidence for 100 data points, in 80 of them the predicted label should match the actual label. I am having a hard time extending this idea to a multi-label scenario where the classifier can predict multiple labels for the utterance each with a probability. What would be a good way to understand it?
I will explain with an example. Let's consider a single label classification problem like sentiment prediction with labels {positive, negative, neutral}. Lets say the data point is I like the movie. The classifier black box outputs (positive, 0.9) (the second value being some sort of a probability value or confidence). We can interpret this as for 100 data points with conf 0.9 in 90 cases the result will be the same as the actual label.
Now, looking at multi label classification, say there is an image with a man sitting on a chair. The actual labels are [man, chair]. The classifier black box outputs these [(man, 0.8), (chair, 0.6), (table, 0.7)], where the second item in the tuple is again some sort of a probability value or confidence. How does a human interpret this?
