2
$\begingroup$

I understand in cases of imbalanced classes in a dataset, accuracy itself is not the best metric as it can be misleading. But in cases of balanced classes, why is precision and recall good metrics? Or what's the advantage of having probabilities?

$\endgroup$
3
  • 4
    $\begingroup$ "But in cases of balanced classes, why is precision and recall good metrics?" - they aren't. $\endgroup$ Mar 28 '18 at 20:30
  • $\begingroup$ I believe F1 and AUC also are good scores for both balanced and unbalanced datasets. $\endgroup$ Mar 29 '18 at 8:21
  • $\begingroup$ F1 score is basically using precision and recall...but i want to know why precision and recall is good for cases where the classes are balanced $\endgroup$
    – Tahm
    Mar 29 '18 at 22:05
1
$\begingroup$
  1. F1-score is preferred only because we know our 'class of interest'.
  2. If we need both the classes in binary classification to be perfectly classified, we will not use f1-score as our measure.
    1. Here probabilities prove advantageous, by plotting an ROC curve, we can visualize and decide, how much sensitivity we can accept sacrificing an amount of sensitivity.
  3. Accuracy performs well with balanced data and considers all the 4 confusion matrix measures, you are right, and you can rely on it only when classes are balanced, might be good for training, but might not be when a new set of data arrives.

Hope this answer helps.

$\endgroup$
0
$\begingroup$

Don't use the F1 score or PR curve in balanced cases, they're not appropiate. They focus on just one class and you won't get a balanced picture.

The most appropiate measure to judge discrimination power for balanced cases, which is also good for unbalanced most of the time is the ROC curve and associated ROC-AUC. PR curves are for needle in haystack situations like credit card fraud, and they also have intrinsic bias owing to the fact they focus only on the predefined positive class. The baseline also varies with datasets with different class distributions so this is why I prefer the precision-recall gain curve.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.