# Confusion Matrix to Calculate Probability

I was asked a relatively simple problem and was curious as to how to solve.

Say I had a bomb detector at the airport, and it is 99.99% correct. That being, when the detector goes off or does not go off, it is correct 99.99% of the time. And in the population, 1/10,000 people actually have a bomb.

What is the probability that when it went off, the person actually had a bomb?

EDIT** This is a Bayes Theorem question. We want to know the probability of a person having a bomb given the false positive/ true positive probability as given here

• The definition of "99.99% correct" is not clear. – user158565 May 18 '17 at 17:56
• You answered your question in the first paragraph: you told us that when the detector goes off, it is correct $99.99\%$ of the time. If that didn't refer to the probability, what does it mean? – whuber May 18 '17 at 18:13
• @whuber if you can take off hold I will answer the problem. – DataTx May 20 '17 at 23:55
• Thanks for clarifying. This is routine bookwork (as is commonly set for homework, for example). Please add the self-study tag, and read its tag-wiki. If you post an answer (as you suggest in comments that you will) that would take the place of showing an attempt. – Glen_b May 21 '17 at 4:58

## 1 Answer

As you claim that 1 in 10000 has a bomb, we can construct following confusion matrix (assuming our population has a size of 10000, WLOG):

           Bomb     NoBomb
Detected   TP          FN
Not Det.   FP          TN
1         9999


Your definition of "correctness" is called Accuracy

$Accuracy = \frac{TP + TN}{TP + TN + FP + FN}= \frac{9999}{10000} = 0.9999$

So mathematically we have 2 variants: either (TP=1, TN=9998), or (TP=0, TN=9999)

Your want to get the value of Precision, that is

$Precision = \frac{TP}{TP + FN}$

So in first case we have the Precision=1, that is the system always detects the bomber, and in the second case, Precision=0 that is system never detects a bomber. And you have no indication which is really your case.

The purpose of this exercise is not to find the value of Precision. It is to show, that in case of highly imbalanced class distribution, the value of Accuracy is not really indicative, and should not be used to measure system performance. In reality, for imbalanced classes, discriminators tend to detect always the major class. So your bomb detection system will never find a bomber, and still will have excellent Accuracy ;)