1
$\begingroup$

I was reading online about generating labels for a dataset manually through clinician review and based on probabilistic models where we get the likelihood.

I was able to read under the advantage of using model based label generation is not just time and money but the below. Let' say my population size is 10K and prevalence of T2DM is 1% which is only 100 people (out of 10k) have T2DM. As extracted from the site.

"it not just saves time and money but, most importantly, it provides sensitivity which clinician chart reviews cannot provide as the acquisition and review of a set of subjects large enough to determine sensitivity is likely not possible. For example, if the prevalence of the health outcome (T2DM) was 1% and the presumed sensitivity was 75%, you would need to review 10K patient records to find 25 false negatives"

population size = 10000 persons

prevalence = 1% which is 100 persons have T2DM

sensitivity = 75%. Meaning out of 100 persons with T2DM, the test/clinician identified only 75 person as with disease.

For false negative, why do they have to review the 10k subjects? Not sure. Can help please?

Can someone help me understand this better in layman terms?

$\endgroup$

1 Answer 1

1
$\begingroup$

The false negatives could have any probability below a given threshold. Especially in a field related to healthcare, these false negatives could cost high. Hence, they need to be reviewed.

Out of 100 bombs, you cannot diffuse 75 of them and be happy that you are safe. You need to diffuse all 100. If your sensitivity is 75% then you need to diffuse all bombs (10K) to get the remaining 25 bombs that your model predicted as 'safe'.

$\endgroup$
3
  • $\begingroup$ Hi, Thanks for the response. Upvoted. $\endgroup$
    – The Great
    Commented Mar 10, 2020 at 14:15
  • $\begingroup$ I am trying to understand this. Must say it's a very good example. Let's say there are 10k bombs and only 100 bombs are set on timer to explode.. Out of this 100, does the above statement in my post mean that the model identified 75 of those bombs as set to explode and needs to be diffused (bomb squad goes in) whereas it failed to identify the remaining 25 bombs. So everyone are happy. But now if Bomb Squad has to manually identify these 25 bombs, then they have to look at all 9925 bombs? Am I right? $\endgroup$
    – The Great
    Commented Mar 10, 2020 at 14:22
  • $\begingroup$ Yes exactly, false negatives are really dependent on the use case. $\endgroup$
    – Danny
    Commented Mar 10, 2020 at 15:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.