Buddy takes a test and tests positive for a disease. Buddy was close to $6$ other friends they all take the same test and end up testing negative. The test has a $FPR=0.01$ and a $FNR=0.15$. What's the percent chance that Buddy was actually negative for the disease?

From the given info, I come up with a confusion matrix where $FP=1$, $TN=99$, $FN=15$, and $TP=85.$

So, is it correct for me to say that the $TPR=0.85$, and that the percent chance Buddy tests negative for the disease is: $$\dfrac{\text{Total Actual Negative}}{\text{Total Actual Negative + Total Actual Positive}} = \dfrac{TN+FP}{(TN+FP)+(FN+TP)} $$

Or would actually just be the $TPR$ here?

  • $\begingroup$ Question for thought: You made an assumption that are equal number of people that actually is positive and negative for the disease respectively. Is this necessarily true? $\endgroup$ – B.Liu Mar 31 at 22:59
  • $\begingroup$ I actually managed to come up a solution that is more on the like of a computer science line of thought (using the confusion matrix, not involving Bayes' theorem explicitly, etc.), I will also put that in later. $\endgroup$ – B.Liu Apr 1 at 10:51
  • $\begingroup$ @B.Liu would appreciate that, as I feel that would really help grasp the nuances of this question! $\endgroup$ – User_13 Apr 1 at 10:57

Let $E_P$, $E_N$, $B_N$, $B_P$ be the events that a tEst comes back positive, a tEst comes back negative, Buddy (or any person) is actually negative, and Buddy (or any person) is actually positive respectively. $E$ is chosen as using $T$ makes it look too similar to TN and TP, which may be involved later.

As provided/inferred from the question:

  • $\mathbb{P}(E_P | B_N) = $ FPR $=0.01$
  • $\mathbb{P}(E_N | B_P) = $ FNR $ = 0.15$
  • $\mathbb{P}(E_P) = 1/7$ (Buddy + 6 friends went for a test, 1 came back positive)

As $E_N$ & $E_P$ are complementary events:

  • $\mathbb{P}(E_P | B_P) = 1 - \mathbb{P}(E_N | B_P) = 0.85$
  • $\mathbb{P}(E_P) = 1 - \mathbb{P}(E_N)$

The question ask for "chance that Buddy was actually negative for the disease", which I take as $\mathbb{P}(B_N | E_P)$. The original interview question setter might as well meant simply $\mathbb{P}(B_N)$ due to lack of pedantry, but we will obtain both of them anyway.

Using Bayes' theorem we have:
$$\mathbb{P}(B_N | E_P) = \frac{\mathbb{P}(E_P | B_N)\mathbb{P}(B_N)}{\mathbb{P}(E_P)}.$$

We know $\mathbb{P}(E_P | B_N)$ and $\mathbb{P}(E_P)$ directly from above, but we need to find $\mathbb{P}(B_N)$. There are two ways to do so:

1. Expanding $\mathbb{P}(E_P)$ using law of total probability

$$ \begin{align} \mathbb{P}(E_P) = \,& \mathbb{P}(E_P | B_N)\mathbb{P}(B_N) + \mathbb{P}(E_P | B_P)\mathbb{P}(B_P) \\ = \,& 0.01 \cdot \mathbb{P}(B_N) + 0.85 \cdot (1-\mathbb{P}(B_N)) \end{align} $$ using quantities and the complementary events laid out above.

As we assumed $\mathbb{P}(E_P) = 1/7$, we can substitute it into the equation above, and rearrange to obtain $\mathbb{P}(B_N) = \frac{0.85 - 1/7}{0.84} \approx 84.18\%$.

2. Form a complementary conditional probability equation

We notice, again by Bayes' theorem, that

$$ \begin{align} \mathbb{P}(B_P | E_P) = \,& \frac{\mathbb{P}(E_P | B_P)\mathbb{P}(B_P)}{\mathbb{P}(E_P)} \\ = \,& \frac{\mathbb{P}(E_P | B_P)(1-\mathbb{P}(B_N))}{\mathbb{P}(E_P)}. \end{align}$$

We know most of the quantities on the LHS, we also know that $\mathbb{P}(B_N | E_P)$ and $\mathbb{P}(B_P | E_P)$ sums to one as they are complementary events, hence for the equation $$ \begin{align} &\, \mathbb{P}(B_N | E_P) + \mathbb{P}(B_P | E_P) \\ = & \frac{\mathbb{P}(E_P | B_N)\mathbb{P}(B_N)}{\mathbb{P}(E_P)} + \frac{\mathbb{P}(E_P | B_P)(1-\mathbb{P}(B_N))}{\mathbb{P}(E_P)}, \end{align}$$ we can solve for $\mathbb{P}(B_N) = \frac{1/7-0.85}{-0.84} \approx 84.18\%$.

Once we know $\mathbb{P}(B_N)$, we can substitute its value back to the very first Bayes' theorem application to obtain $$\mathbb{P}(B_N | E_P) = \frac{0.01 \cdot \frac{0.85-1/7}{0.84}}{1/7}\approx 5.89\%.$$

  • $\begingroup$ I do have to say that this is a really innocent-looking interview question with a very sharp sting in the tail. Might steal it for some occasion in the future! $\endgroup$ – B.Liu Apr 1 at 10:10
  • $\begingroup$ I just realized, isn't the 6 friends thing in the question just some unnecessary info to trick you? It seems like that. What was your logic behind 1/7 in this case? $\endgroup$ – User_13 Apr 1 at 21:27
  • $\begingroup$ @User_13 I believe that piece of info is necessary to make the question returning just one possible answer. The 1/7 in the answer is a reflection of that info - 7 people went for a test and 1 came back testing positive. $\endgroup$ – B.Liu Apr 2 at 5:31

This is an answer more in line with the "Computer Science perspective", in which it utilises the confusion matrix quantities and systems of equations, but not Bayes' theorem explicitly.

Consider the usual $2 \times 2$ confusion matrix, and define the positive event as "having the disease". We denote the matrix elements TP, FP, FN, TN, corresponding to the true positives, false positives, false negatives, and true negatives. The question gave us the following information:

  1. FPR $= \frac{FP}{FP+TN} = 0.01$
  2. FNR $= \frac{FN}{FN+TP} = 0.15$
  3. Test positivity odds $= \frac{\textrm{Tested }P}{\textrm{Tested }N} = \frac{TP+FP}{FN+TN} = 1/6$

The question asks us to find Buddy is actually negative given a positive test, i.e. $\frac{FP}{TP+FP}$.

We can rearrange the first information to obtain $TN = 99 \cdot FP$. Similarly, we can rearrange the second information to get $FN = 0.15 / 0.85 \cdot TP$. Substituting the two rearranged equation to the third information, we have

$$ \begin{align}& \frac{TP + FP}{\frac{0.15}{0.85} \cdot TP + 99 \cdot FP} = \frac{1}{6} \\ \iff & TP = \frac{\frac{99}{6} - 1}{1 - \frac{0.15}{6 \times 0.85}} \cdot FP. \end{align}$$

Substituting this relation into our required quantity we have:

$$ \frac{FP}{TP + FP} = \frac{FP}{\frac{\frac{99}{6} - 1}{1 - \frac{0.15}{6 \times 0.85}} \cdot FP + FP} = \frac{1}{\frac{\frac{99}{6} - 1}{1 - \frac{0.15}{6 \times 0.85}} + 1}, $$ which evaluates to around 5.89%, and checks out with what we obtained with the Bayes' theorem approach.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.