Confusion Matrix to Calculate Probability I was asked a relatively simple problem and was curious as to how to solve.
Say I had a bomb detector at the airport, and it is 99.99% correct. That being, when the detector goes off or does not go off, it is correct 99.99% of the time. And in the population, 1/10,000 people actually have a bomb.
What is the probability that when it went off, the person actually had a bomb?
EDIT** 
This is a Bayes Theorem question. We want to know the probability of a person having a bomb given the false positive/ true positive probability as given here
 A: As you claim that 1 in 10000 has a bomb, we  can construct following confusion matrix (assuming our population has a size of 10000, WLOG):
           Bomb     NoBomb
Detected   TP          FN   
Not Det.   FP          TN
            1         9999

Your definition of "correctness" is called Accuracy
$ Accuracy =  \frac{TP + TN}{TP + TN + FP + FN}= \frac{9999}{10000} = 0.9999 $
So mathematically we have 2 variants: either (TP=1, TN=9998), or (TP=0, TN=9999)
Your want to get the value of Precision, that is
$ Precision = \frac{TP}{TP + FN} $
So in first case we have the Precision=1, that is the system always detects the bomber, and in the second case, Precision=0 that is system never detects a bomber. And you have no indication which is really your case. 
The purpose of this exercise is not to find the value of Precision. It is to show, that in case of highly imbalanced class distribution, the value of Accuracy is not really indicative, and should not be used to measure system performance. In reality, for imbalanced classes, discriminators tend to detect always the major class. So your bomb detection system will never find a bomber, and still will have excellent Accuracy ;)  
A: There's indeed a relationship Confusion Matrix to Bayes:
Bayes’ theorem is crucial for interpreting the results from binary classification algorithms, I will show that Bayes’ theorem is simply the relationship between precision and recall:
Suppose we have A and B case as 0 and 1.
print(confusion_matrix(y_test, y_pred))
print(classification_report(y_test, y_pred))
dtype: int64
[[ 39 32]   71 =A
 [ 22 222]] 244 =B
  254 315

             Predict
Actual 0  [[ 39  32]    71
       1   [ 22 222]]  244
             61 254    315 

1 precision: 222/254 =0.874 P(A/B)
2 recall: 222/244 =0.909 = Bayes P(B/A)
So the recall (sensitivity) results in Bayes P(B/A)
The probability P(B/A) = 222/244 = 0.91 is called the recall. It simply gives the percentage of the 244 actual B that were correctly predicted by our classification algorithm. We see that Bayes’ theorem is simply a relationship between recall and precision:
[Predict * precision / Actual = recall] = Bayes
(P_B*P_AB)/P_A = 0.9098360655737705
(P(B)*P(A/B))/(P(A)) = 0.9098 = P(B/A)
(254/315 * 222/254) / (244/315) = 0.9098
predict * precision / actual = recall
prior * likelihood / evidence = posterior
precision / recall = actual / predict
      precision    recall  f1-score   support

   0       0.64      0.55      0.59        71
   1       0.87      0.91      0.89       244

accuracy                           0.83       315
You can explore that more with a forked jupyter notebook in python:
https://github.com/maxkleiner/Bayes_theorem/blob/master/Bayes_Theorem.ipynb
