I am running a CNN image multi-class classification model with Keras/Tensorflow and have established about a 90% overall accuracy with my best model trial. I have 10 unique classes I am trying to classify. However I want to present a PRC for the individual classes. I am trying to wrap my head around the intuition behind using thresholds for each class.

Since I utilize a SoftMax output layer activation I end up with a probability distribution for the 10 class label possibilities. If I am to create a PRC for this, I utilize different thresholds for each individual class and then classify the image as positive or negative depending on the probability.

Now in reality suppose I was using a threshold of 0.4 and my softmax activation gave me P(dog) = .41 and P(cat) = .42 and then small probabilities for the other classes. Thus with this threshold I am confirming my dog image as a dog even though the softmax probability was higher for cat.

Can someone explain to me how this makes sense in practice (if I were to deploy a classification model like this). My model would of course classify this image as a cat in practice since softmax chooses class with highest probability. What exactly is my PRC actually helping with when presenting evaluation metrics? Is it simply just showing how an individual class performs in the binary sense with a threshold for classification? In practice it wouldn't be using these thresholds for actual prediction.

Here is an example of what my PRC looks like for my validation set.

enter image description here



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.