Estimating positive and negative predictive value without knowing the prevalence

Question

There is a lot of discussion about the positive predictive value of a test currently. I know that if I know specificity, sensitivity of a test and the prevalence $p$ in the sample, then I can easily calculate the positive predictive value (ppv) and negative predictive value (npv):

$ppv = \frac{p\cdot Sens}{p\cdot Sens + (1-p)\cdot(1-Spec)}$

and

$npv = \frac{(1-p)\cdot Spec}{(1-p)\cdot Spec+p\cdot(1-Sens)}$.

However, this requires that I know the prevalence, and of course the only way knowing this number is from the test, for which I don't know the ppv etc...

However, I was wondering if it shouldn't be possible to also use the positive proportion and the number of tests instead to estimate the ppv and the npv in a Bayesian framework. The thinking is like this:

Given sufficiently high specificity and sensitivity, if I have 90 positive out of 100 total tests, it is highly unlikely that all these tests are false positives. Even 80 false positive tests seem very unlikely, if I assume for example a specificity of 95%:

• There are at most 90 negative cases
• Probability for seeing 80 out of 90 at 5%: 2.833227e-92

So, such a low ppv is just not consistent with the observation. This led me to the following model in JAGS:

rm(list = ls())

#### Model Parameters
N     <- 360139 # Number of Sars-CoV2 tests week 15 in Germany
N.pos <-  29302 # Number of positives tests in week 15
Spec  <- .956
Sens  <- .989

lim.min <- 0.00001
lim.max <- 0.99999

#### Sampler parameters
n.iter <- 100000
n.burn <-  10000
n.chains <- 8
n.thin   <- 4


library(R2jags)

modelstring <- "
model {
  # Probability of being infected
  p.inf ~ dbeta(1,1) T(lim.min, lim.max)
  # Number of infected among the tested
  N.inf ~ dbin(p.inf, N)
  # Not infected is the rest of the test
  N.ninf <- N - N.inf
  # number of true positives
  N.tpos ~ dbin(Sens, N.inf)
  # False positives based on uninfected in the sample
  N.fpos ~ dbin(1-Spec, N.ninf)
  
  ppv <- N.tpos / (N.pos)
  npv <- N.tneg / (N.tneg + N.fneg)

  # Just for outpout
  N.tneg <- N.ninf - N.fpos
  N.fneg <- N.inf  - N.tpos

  # True and false positives have to make up the
  # total number of positive tests (sum of binomials)
  N.pos ~ sum(N.tpos, N.fpos)
}
"

init <- function(){
  nn <- ( N.pos / 2 )
  list( 
    N.tpos = nn,
    N.fpos = N.pos - nn,
    N.inf  = nn
  )
}

jData <- list(
  lim.min = lim.min,
  lim.max = lim.max,
  
  N      = N,
  N.pos  = N.pos,
  Spec   = Spec,
  Sens   = Sens
)

params <- c("p.inf","N.inf","N.tpos","N.fpos","N.tneg","N.fneg","ppv","npv")

jres <- jags(data=jData, inits = init, model.file = textConnection(modelstring), parameters.to.save = params, 
             n.thin = n.thin, n.iter = n.iter, n.burnin = n.burn, n.chains = n.chains )
jres

Is my thinking correct, and could such a model estimate the ppv and npv without any assumptions about the actual prevalence (flat prior at the p.inf variable). Is such an approach used in practice as well, or is the actual ppv estimated differently?

I think it should be possible to define a maximum likelihood version of this model as well, but due to the sum of binomials it will probably be very ugly.

Answer 1

I don't use RJags so I can't confirm your code but I would say 'yes' your idea makes sense with three caveats:

First, (intuitively) your likelihood contains little-to-no information on the prevalence parameter, and so your posterior will be relying nearly exclusively on the prior specification. I would recommend doing sensitivity (different meaning of that word here!) analyses to different choices of prior distributions for the prevalence parameter or different fixed values of the prevalence.

Second, even though you put sensitivity and specificity under the header #### Model Parameters in your script, you are actually treating sensitivity and specificity as fixed and known data, by my reading. I would say they should be treated as genuine parameters, i.e. equipped with prior distributions. When prevalence is low, then ppv will be extremely sensitive (yet another different use of the word!) to small changes in the specificity; conversely, if the prevalence is large, then npv will be extremely sensitive to small changes in the sensitivity. Andrew Gelman wrote a blog post about the controversial Stanford study that was critical of the analysis for, among other things, not incorporating the substantial uncertainty about estimates of the specificity for covid tests.

Third, I'm wondering about your sampling design. Does it make sense to think about a single prevalence in your context? I presume in Germany (where your data come from, based on the comment in your script) that things a bit more systematic than the US (where I live). However, I would be concerned that some folks are pursuing tests because they have symptoms or were exposed (and so the decision to test is likely correlated with the outcome of the test), whereas other folks may be tested for external reasons, i..e they need to do so in order to return to work or visit an elderly relative (and so the decision to test is not correlated with the outcome).