1
$\begingroup$

There is this question on the F-1 score, asking why we compute the harmonic mean of precision and recall rather than its arithmetic mean. There were good arguments in the answers in favor of the harmonic mean, in particular that it is suited to take the average of ratios and drops to zero whenever one of the other does.

Which begs the question, why is the harmonic mean of sensitivity and specificity not a thing (to my knowledge)? There are both ratios and the same fine arguments could apply.

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ Of possible interest: Academic reference on the drawbacks of accuracy, F1 score, sensitivity and/or specificity $\endgroup$
    – Dave
    Commented Mar 22, 2023 at 21:47
  • $\begingroup$ Specificity, as a KPI to be optimized, presupposes a very specific cost structure (as well as that there are only two actions possible, a heroic assumption). Sensitivity assumes a different, but still specific cost structure. Optimizing the harmonic mean assumes yet a third cost structure. Compare this. It is far better to create probabilistic classifications and cleanly separate the modeling from the decision aspect, where costs of possible actions enter in the decision-making step. See here. $\endgroup$
    – Stephan Kolassa
    Commented Mar 23, 2023 at 7:20
  • $\begingroup$ I understand that harmonic and arithmetic means assume different cost structure. My question is why they are consistently chosen differently for the se/sp and p/r pairs. Why the difference in treatment, what is the fundamental difference between the two that underlies this? $\endgroup$
    – user209974
    Commented Mar 23, 2023 at 10:15

2 Answers 2

Reset to default
0
$\begingroup$

I feel like the question should be "why is X a thing" rather than "why is X not a thing." Sure we can calculate the harmonic mean (or any other mean) of sensitivity and specificity, but why would we want to this? What would this mean mean (sorry for the pun #notsorry)?

Sensitivity and specificity are separate concepts in that you can have one without the other, and which you prefer depends on your context. Suppose you want early detection of a deadly disease in the population, and your strategy is to start with a cheap screening test for everyone to identify a high risk subgroup, who go on to have a costly test to confirm the diagnosis. Sensitivity is more important for the former and specificity is more important for the latter.

If for some reason you wish to compromise between sensitivity and specificity, note that the compromise is usually non-linear. Suppose your test has 100% sensitivity and 10% specificity and you're willing to lose 5% sensitivity to gain specificity. You can just have the same test, but set the threshold of a positive result to be more stringent. But how much specificity you gain by spending 5% sensitivity in this way depends on how good the test is as per its "receiver operating characteristic (ROC)" curve, and one way of optimising along this curve could be via "Youden's J index." If your test has a high area under the ROC curve (AUC), you might end up with 95% sensitivity and 99% specificity. If your test has low ROC AUC, you might only get 95% sensitivity and 11% specificity, and even though you keep spending sensitivity, you never get better than 60% sensitivity and 60% specificity. Meanwhile (#notsorry), the mean of sensitivity and specificity never comes up in all this discussion... so why would anyone care about it?

Improve this answer
$\endgroup$
2
  • $\begingroup$ No, my question was indeed on "Why is X not a thing". Your answer on why balanced accuracy makes sense says nothing about why harmonic mean doesn't, or does less. Besides, I could replace specificity and sensitivity in your answer with precision and recall, and all the arguments would still hold, so it doesn't help in explaining why those two pairs get a different treatment. $\endgroup$
    – user209974
    Commented Mar 23, 2023 at 9:27
  • $\begingroup$ I now realise that you're perhaps asking in a machine learning context rather than say a medical diagnosis context, and I think they have different goals and limitations (in medicine, balanced accuracy is essentially "not a thing" so harmonic accuracy wouldn't be either). Two things I wanted to clarify: 1) My answer was not on "why balanced accuracy makes sense." I never mention balanced accuracy at all. 2) Specificity and sensitivity do not correspond to precision and recall. Recall is analogous to sensitivity, but precision is analogous to positive predictive value rather than specificity. $\endgroup$
    – Singularity24601
    Commented Mar 23, 2023 at 10:38
0
$\begingroup$

Attempt #2, with a different focus as I suspect this is more a machine learning context rather than a healthcare context. I'm still going to reinterpret your question as "why might we prefer an arithmetic mean over a harmonic mean for sensitivity and specificity?" because it's far easier to suggest why X is a thing rather than why X is not a thing, since I have to consider one case with the former and nigh infinite cases with the latter.

Suppose I'm deciding whether 1000 people have cancer, but I win $1 for each correct guess and I want to maximise money. Unlike positive predictive value, sensitivity and specificity and purely measures of how good the *test* is and are independent of the *data.* Suppose I want a test that will be good regardless of the data because I have no idea whether 90% or 10% of the 1000 have cancer. If I use a test with 80% sensitivity and 30% specificity (higher arithmetic mean), I win either $350 or $750, average $550. If I use a test with 50% sensitivity and 50% specificity (higher harmonic mean), I win $500.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.