What is the advantage of using precision and recall metrics in classification?

Question

I understand in cases of imbalanced classes in a dataset, accuracy itself is not the best metric as it can be misleading. But in cases of balanced classes, why is precision and recall good metrics? Or what's the advantage of having probabilities?

Answer 1

1. F1-score is preferred only because we know our 'class of interest'.
2. If we need both the classes in binary classification to be perfectly classified, we will not use f1-score as our measure.
   3. Here probabilities prove advantageous, by plotting an ROC curve, we can visualize and decide, how much sensitivity we can accept sacrificing an amount of sensitivity.
3. Accuracy performs well with balanced data and considers all the 4 confusion matrix measures, you are right, and you can rely on it only when classes are balanced, might be good for training, but might not be when a new set of data arrives.

Hope this answer helps.

Answer 2

Don't use the F1 score or PR curve in balanced cases, they're not appropiate. They focus on just one class and you won't get a balanced picture.

The most appropiate measure to judge discrimination power for balanced cases, which is also good for unbalanced most of the time is the ROC curve and associated ROC-AUC. PR curves are for needle in haystack situations like credit card fraud, and they also have intrinsic bias owing to the fact they focus only on the predefined positive class. The baseline also varies with datasets with different class distributions so this is why I prefer the precision-recall gain curve.