I was asked a relatively simple problem and was curious as to how to solve.

Say I had a bomb detector at the airport, and it is 99.99% correct. That being, when the detector goes off or does not go off, it is correct 99.99% of the time. And in the population, 1/10,000 people actually have a bomb.

What is the probability that when it went off, the person actually had a bomb?

EDIT** This is a Bayes Theorem question. We want to know the probability of a person having a bomb given the false positive/ true positive probability as given here

  • 1
    $\begingroup$ The definition of "99.99% correct" is not clear. $\endgroup$ – user158565 May 18 '17 at 17:56
  • $\begingroup$ You answered your question in the first paragraph: you told us that when the detector goes off, it is correct $99.99\%$ of the time. If that didn't refer to the probability, what does it mean? $\endgroup$ – whuber May 18 '17 at 18:13
  • $\begingroup$ @whuber if you can take off hold I will answer the problem. $\endgroup$ – DataTx May 20 '17 at 23:55
  • $\begingroup$ Thanks for clarifying. This is routine bookwork (as is commonly set for homework, for example). Please add the self-study tag, and read its tag-wiki. If you post an answer (as you suggest in comments that you will) that would take the place of showing an attempt. $\endgroup$ – Glen_b May 21 '17 at 4:58

As you claim that 1 in 10000 has a bomb, we can construct following confusion matrix (assuming our population has a size of 10000, WLOG):

           Bomb     NoBomb
Detected   TP          FN   
Not Det.   FP          TN
            1         9999

Your definition of "correctness" is called Accuracy

$ Accuracy = \frac{TP + TN}{TP + TN + FP + FN}= \frac{9999}{10000} = 0.9999 $

So mathematically we have 2 variants: either (TP=1, TN=9998), or (TP=0, TN=9999)

Your want to get the value of Precision, that is

$ Precision = \frac{TP}{TP + FN} $

So in first case we have the Precision=1, that is the system always detects the bomber, and in the second case, Precision=0 that is system never detects a bomber. And you have no indication which is really your case.

The purpose of this exercise is not to find the value of Precision. It is to show, that in case of highly imbalanced class distribution, the value of Accuracy is not really indicative, and should not be used to measure system performance. In reality, for imbalanced classes, discriminators tend to detect always the major class. So your bomb detection system will never find a bomber, and still will have excellent Accuracy ;)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.