# Estimating positive and negative predictive value without knowing the prevalence

There is a lot of discussion about the positive predictive value of a test currently. I know that if I know specificity, sensitivity of a test and the prevalence $$p$$ in the sample, then I can easily calculate the positive predictive value (ppv) and negative predictive value (npv):

$$ppv = \frac{p\cdot Sens}{p\cdot Sens + (1-p)\cdot(1-Spec)}$$

and

$$npv = \frac{(1-p)\cdot Spec}{(1-p)\cdot Spec+p\cdot(1-Sens)}$$.

However, this requires that I know the prevalence, and of course the only way knowing this number is from the test, for which I don't know the ppv etc...

However, I was wondering if it shouldn't be possible to also use the positive proportion and the number of tests instead to estimate the ppv and the npv in a Bayesian framework. The thinking is like this:

Given sufficiently high specificity and sensitivity, if I have 90 positive out of 100 total tests, it is highly unlikely that all these tests are false positives. Even 80 false positive tests seem very unlikely, if I assume for example a specificity of 95%:

• There are at most 90 negative cases
• Probability for seeing 80 out of 90 at 5%: 2.833227e-92

So, such a low ppv is just not consistent with the observation. This led me to the following model in JAGS:

rm(list = ls())

#### Model Parameters
N     <- 360139 # Number of Sars-CoV2 tests week 15 in Germany
N.pos <-  29302 # Number of positives tests in week 15
Spec  <- .956
Sens  <- .989

lim.min <- 0.00001
lim.max <- 0.99999

#### Sampler parameters
n.iter <- 100000
n.burn <-  10000
n.chains <- 8
n.thin   <- 4

library(R2jags)

modelstring <- "
model {
# Probability of being infected
p.inf ~ dbeta(1,1) T(lim.min, lim.max)
# Number of infected among the tested
N.inf ~ dbin(p.inf, N)
# Not infected is the rest of the test
N.ninf <- N - N.inf
# number of true positives
N.tpos ~ dbin(Sens, N.inf)
# False positives based on uninfected in the sample
N.fpos ~ dbin(1-Spec, N.ninf)

ppv <- N.tpos / (N.pos)
npv <- N.tneg / (N.tneg + N.fneg)

# Just for outpout
N.tneg <- N.ninf - N.fpos
N.fneg <- N.inf  - N.tpos

# True and false positives have to make up the
# total number of positive tests (sum of binomials)
N.pos ~ sum(N.tpos, N.fpos)
}
"

init <- function(){
nn <- ( N.pos / 2 )
list(
N.tpos = nn,
N.fpos = N.pos - nn,
N.inf  = nn
)
}

jData <- list(
lim.min = lim.min,
lim.max = lim.max,

N      = N,
N.pos  = N.pos,
Spec   = Spec,
Sens   = Sens
)

params <- c("p.inf","N.inf","N.tpos","N.fpos","N.tneg","N.fneg","ppv","npv")

jres <- jags(data=jData, inits = init, model.file = textConnection(modelstring), parameters.to.save = params,
n.thin = n.thin, n.iter = n.iter, n.burnin = n.burn, n.chains = n.chains )
jres


Is my thinking correct, and could such a model estimate the ppv and npv without any assumptions about the actual prevalence (flat prior at the p.inf variable). Is such an approach used in practice as well, or is the actual ppv estimated differently?

I think it should be possible to define a maximum likelihood version of this model as well, but due to the sum of binomials it will probably be very ugly.

Second, even though you put sensitivity and specificity under the header #### Model Parameters in your script, you are actually treating sensitivity and specificity as fixed and known data, by my reading. I would say they should be treated as genuine parameters, i.e. equipped with prior distributions. When prevalence is low, then ppv will be extremely sensitive (yet another different use of the word!) to small changes in the specificity; conversely, if the prevalence is large, then npv will be extremely sensitive to small changes in the sensitivity. Andrew Gelman wrote a blog post about the controversial Stanford study that was critical of the analysis for, among other things, not incorporating the substantial uncertainty about estimates of the specificity for covid tests.