0
$\begingroup$

I've been told by my professor that hit rate is more important when it comes to real-life deployment but sensitivity and other accuracy rates need to be satisfactory also. However, my other professor also says that sometimes sensitivity is preferred over hit rate? What's the big difference about it?

$\endgroup$
0
$\begingroup$

All KPIs in classification can be important. (See the Wikipedia article on sensitivity and specificity for the entire zoo of common KPIs.) Which one is more important depends on the costs of correct and wrong classifications. Sometimes false positives are more costly than false negatives, or vice versa, and the relationships between these costs (and those of correct classifications) drive what we are most interested in.

I have been arguing at great length that these KPIs, which try to optimize the statistical/machine learning aspect of the pipeline through a focus on the decision, which is a different part of the pipeline than the stats/ML one, is fundamentally misguided.

Instead, we should be aiming for calibrated and sharp probabilistic classifications and assess these using proper scoring rules. Then, and only then, can we discuss what the optimal decision to take is when faced with a particular probabilistic classification - a discussion that should include the costs of correct and wrong decisions, of course.

See my answer noted above as well as my answer to Why is accuracy not the best measure for assessing classification models? - note that every single criticism that is leveled there against accuracy applies equally to sensitivity, specificity and the other KPIs (AUROC is a slightly different case).

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.