What is the standard error of a binomial process with a false-positive rate

Question

I have a question that must be common, but after searching, I can't find an explanation. Suppose we wish to estimate the percentage of people, p, who have a disease. We test n people. The test has known true-positive and a false-positive rates of TP and FP. The expected number positive tests is $E[Y]=TPp+FP(1-p)$. Solving for p, the estimate of p is $$p = \frac{(y/n)-FP}{TP-FP}$$ How do we calculate the error in the estimate of p? There is uncertainty/error from the sampling of the population (binomial distribution of with n samples and probability p) and additional uncertainty/error from the test's true-positives and false-positive rates, which are also binomial distributions.

Answer 1

Let $T$=TP and $F$=FP, which are known and constant. Also assume $T\neq F$.

Taking a step back, let $Y_i$ be an independent binary variable corresponding to a single patient's test result. We have \begin{align*} P(Y_i=1)&=Tp+F(1-p)=F+(T-F)p\\ P(Y_i=0)&=1-F-(T-F)p\\ \end{align*} The variance (as $Y_i$ is a Bernoulli variable) is thus $$Var(Y_i)=\Big(F+(T-F)p\Big)\Big(1-F-(T-F)p\Big)$$

Now, turning our attention to $\hat{p}$, the estimator for $p$. As you say, it is: \begin{align*} \hat{p}&=\frac{y-F}{T-F} \end{align*}

where we defined $y:=\frac{1}{N}\sum_i^N Y_i$, which is the overall percentage of positive test results. The variance of $\hat{p}$ is

\begin{align*} Var(\hat{p})&=\frac{\sum_i^N Var(Y_i)}{N^2(T-F)^2}\\ &= \frac{Var(Y_i)}{N(T-F)^2}\\ &= \frac{\Big(F+(T-F)p\Big)\Big(1-F-(T-F)p\Big)}{N(T-F)^2} \end{align*}

The standard error is the square root of the above, while substituting in our estimate $\hat{p}$:

\begin{align*} SE(\hat{p})&=\sqrt{\frac{\Big(F+(T-F)\hat{p}\Big)\Big(1-F-(T-F)\hat{p}\Big)}{N(T-F)^2}}\\ &=\sqrt{\frac{y(1-y)}{N(T-F)^2}} \end{align*}

This expression is consistent with our intuition that the standard error is small (ie. high certainty in our estimation of the prevalence $p$) when $N$ is large, $T$ is large or $F$ is small.

---

Footnote: what if $T=F$? Intuitively, this means the test is worthless; sick and healthy patients are just as likely to test positive, meaning the test is completely uncorrelated to the disease process. Hence there's no point trying to estimate $p$ from such a test.