1
$\begingroup$

So let's say we have a multi-class classification problem and we want to represent the outcomes as confusion matrix. All the examples I'm finding on the web asume that all the elements are detected. I understand the idea of the confusion matrix but I can't see how one can represent the non-detected elements on it.

$\endgroup$
0
0
$\begingroup$

You very well may want to abstain from classifying certain examples. It may be more costly to classify incorrectly than to state that you are uncertain and unwilling to classify, for example in a recommendation system.

Consider non-detection a new class that you never observe in the training data. You can build in a penalty for non-detection in your loss function, if you want to start at the very beginning. However, a simpler solution is just to set a probability cutoff beneath which you abstain from classifying--e.g. if the vector of estimated class membership probabilities has maximum less than 60%, don't classify. As long as the loss function is symmetric in the various classes, this corresponds to a implicit penalty on misclassification. You can even set a threshold for each class if failing to classify has higher/lower cost for certain classes.

$\endgroup$
1
  • $\begingroup$ But the q asks how can you then define/calculate a confusion matrix? Can you answer that? $\endgroup$ – kjetil b halvorsen Sep 26 '19 at 6:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.