2
$\begingroup$

There are two Covid-19 tests

  • Test A: Sensitivity 92%, Specificity 99.7%
  • Test B: Sensitivity 97%, Specificity 99.7%

Assume a person takes three tests consecutively (i.e. right after another), in the order A, A, B.

The results are as follows:

  • First time (using Test A): Positive
  • Second time (using Test A): Positive
  • Third time (using Test B): Negative

I've tried to calculate the probability that person is actually positive, but I'm making a mistake somewhere.

Can someone tell me how to calculate this? I'm not sure if additional information would be needed, such as the Covid-19 incidence?

Make any simplifiying assumptions you need, and assume the test results are independent of each other.

$\endgroup$
8
  • 2
    $\begingroup$ It cannot be calculated without making strong simplifying assumptions, which likely are false. In particular, when one kind of test gives a mistaken result for a particular patient, perhaps it will consistently produce the same error on retests of the same patient. It is possible the errors made by two different tests could be correlated (either positively or negatively), too. $\endgroup$
    – whuber
    Jan 17, 2022 at 18:19
  • 1
    $\begingroup$ Ok, understood :). I edited the question to say that any simplifying assumptions should be made, and tests should be assumed as independent. Does that work? $\endgroup$
    – ChrisC
    Jan 17, 2022 at 18:22
  • 2
    $\begingroup$ Yes it does, thank you. (You evidently also need to assume the patient's status is the same for all three tests; that is, either they were definitely infected or definitely not infected at the times each of the tests were taken.) $\endgroup$
    – whuber
    Jan 17, 2022 at 18:29
  • $\begingroup$ Ah ok, that’s what I thought. Not sure what would be a sensible (of course grossly simplified, but somewhat sensible under simplified assumptions) probability. This is for Germany. In the past 7 days an approximate average of 810.000 people have had ongoing (i.e. „active“ infections) according to statista de.statista.com/statistik/daten/studie/1181971/umfrage/…. Let’s assume that is accurate (which it might not be), would it be somewhat sensible to divide that average of 810.000 active infections by Germany‘s population of… $\endgroup$
    – ChrisC
    Jan 17, 2022 at 18:50
  • 1
    $\begingroup$ You do need the ratio of infected to total population, see answer below. $\endgroup$ Jan 17, 2022 at 19:15

1 Answer 1

1
$\begingroup$

Notation:

The results of the tests are $T_1,T_2,T_3 \in \{0,1\}$. We indicate a positive test by $T_i = 1$ and zero otherwise, for $i \in \{1,2,3\}$. Similarly, suppose that $Y$ is the status of having covid $(Y=1)$ or not $(Y=0)$.

The accuracy of each test is given by:

  • The sensitivity, $\alpha_i = \mathbb{P}(T_i = 1 \mid Y=1)$
  • The specificity, $\beta_i = \mathbb{P}(T_i = 0 \mid Y = 0)$.

Problem:

We want to compute the conditional likelihood of infection $(p)$, which is equal to $$ p = \mathbb{P}(Y=1 \mid T_1 = 1, T_2 = 1, T_3 = 0) $$ For ease of notation let $\mathcal{T}$ denote the event that the tests have two positives and one negative. Then $$ p = \mathbb{P}(Y = 1 \mid \mathcal{T}). $$

A Bayes' approach:

We can write the probability as: $$ p = \frac{\mathbb{P}(\mathcal{T} \mid Y = 1)\mathbb{P}(Y = 1)}{\mathbb{P}(\mathcal{T})} $$ The value $\mathbb{P}(Y = 1)$ is the unconditional probability that somebody has COVID, and $\mathbb{P}(\mathcal{T})$ is the probability of observing a set of testing results in the population. The value $\mathbb{P}(\mathcal{T} \mid Y = 1)$ is the likelihood of a observing a test sequence for an infected person.

We need assumptions to figure out each quantity:

  • Assumption 1: $\mathbb{P}(Y = 1)$ is equal to the local covid infection rate.
  • Assumption 2: The tests $T_1,T_2,T_3$ are independent given $Y$ (the true status)

$$ \mathbb{P}(T_1 = t_1,T_2 = t_2, T_3 = t_3 \mid Y =y) = \prod_{i=1}^3 \mathbb{P}(T_i = t_i \mid Y = y). $$

where $(t_1,t_2,t_3)$ are the values of the test for a specific patient.

Sketch of the computation steps:

First you need to determine the local infection rate (which is not given by your problem). Then you need use the Bayes' formula for $p$.

  • You can compute $\mathbb{P}(\mathcal{T} \mid Y = 1)$ by using Assumption 2, and using the appropriate value of sensitivity and specificity for each test. For example, for the third test you need to compute $1-\alpha_3$ to obtain the likelihood of a negative test.
  • You can compute $\mathbb{P}(\mathcal{T})$ by the relationship: $$ \mathbb{P}(\mathcal{T}) = \mathbb{P}(\mathcal{T} \mid Y = 1)\mathbb{P}(Y = 1) + \mathbb{P}(\mathcal{T} \mid Y = 0)\mathbb{P}(Y=0)$$ In Step 2 you have to be careful keeping track of which probabilities you need, and how you can obtain them from $\{\alpha_i,\beta_i)\}_{i=1}^{3}$. Plugging in the values of $(\alpha_i,\beta_i)$ without thinking about the specific sequence of test results in your problem is a potential pitfall.
$\endgroup$
8
  • $\begingroup$ Cool. I'm trying to do the calculations right now, but am encountering some issues. Specifically, I am trying to calculate P(T3 = 0). Here is my process so far: P(T3 = 0) = P(T3 = Negative) = P(T3 = False Negative) + P(T3 = True Negative) = (1-Sensitivity of T3) + Specifity of T3. Plugging in the numbers, the leftmost side of this equation gives, using the values for sensitivity and specifity for Test B: (1-0.97) + 0.997 = 1.027. Not sure if I'm on the wrong track completely, but a probablity larger 1 strikes me as odd :) Edit: think I found my mistake, trying now $\endgroup$
    – ChrisC
    Jan 17, 2022 at 22:42
  • $\begingroup$ I believe my mistake in my previous comment is trying to calculate, for example, P(T3 = False Negative) as 1- Sensitivity of T3. It would probably be [1- (Sensitivity of T3) - Specifity of T3 - (1- Specifity of T3)]? But I'm not sure if I can work that out directly. Maybe that's why you suggested using Bayes' Formula, which I have no experience with. Might you guide me in solving this numerically? :) $\endgroup$
    – ChrisC
    Jan 17, 2022 at 22:49
  • 1
    $\begingroup$ In your first comment you need to weigh each probability by $\mathbb{P}(Y=1)$ and $\mathbb{P}(Y= 0)$, respectively. This ensures that the result is always in $[0,1]$. This is known as the law of total probability. Intuitively, it says that you need to weight the false negatives and true negatives by the likelihood that you have covid to begin with, en.wikipedia.org/wiki/Law_of_total_probability. $\endgroup$ Jan 17, 2022 at 22:59
  • 1
    $\begingroup$ I encourage you to work through the problem in the solution because all the formulas to solve it are there, no need to bring other knowledge. I used Bayes' formula to produce the equation for $p$, but beyond that it's all self-contained. $\endgroup$ Jan 17, 2022 at 23:01
  • 1
    $\begingroup$ For part 1 of the computation, you need to multiply together the probabilities of observing each test (assuming that the patient has covid: use only the sensitivity parameters). Before doing part 2 you need to do the same thing assuming the patient doesn't have covid. To do part 2, use the law of total probability as indicated in my comments above. $\endgroup$ Jan 17, 2022 at 23:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.