# What is the standard error of a binomial process with a false-positive rate

I have a question that must be common, but after searching, I can't find an explanation. Suppose we wish to estimate the percentage of people, p, who have a disease. We test n people. The test has known true-positive and a false-positive rates of TP and FP. The expected number positive tests is $$E[Y]=TPp+FP(1-p)$$. Solving for p, the estimate of p is $$p = \frac{(y/n)-FP}{TP-FP}$$ How do we calculate the error in the estimate of p? There is uncertainty/error from the sampling of the population (binomial distribution of with n samples and probability p) and additional uncertainty/error from the test's true-positives and false-positive rates, which are also binomial distributions.

• Could you clarify if $TP,FP$ are to be treated as known or unknown? You state them as known, but are concerned with the uncertainty in them later. Commented Jan 28 at 14:24
• @user1848065 I think OP meant that any single patient not only undergoes a Bernoulli trial to determine their healthy/sick status, but also a second trial to determine whether their test flags positive, which depends on TP and FP. I don't think OP meant that the TP and FP values themselves have uncertainty Commented Jan 28 at 23:07

Let $$T$$=TP and $$F$$=FP, which are known and constant. Also assume $$T\neq F$$.

Taking a step back, let $$Y_i$$ be an independent binary variable corresponding to a single patient's test result. We have \begin{align*} P(Y_i=1)&=Tp+F(1-p)=F+(T-F)p\\ P(Y_i=0)&=1-F-(T-F)p\\ \end{align*} The variance (as $$Y_i$$ is a Bernoulli variable) is thus $$Var(Y_i)=\Big(F+(T-F)p\Big)\Big(1-F-(T-F)p\Big)$$

Now, turning our attention to $$\hat{p}$$, the estimator for $$p$$. As you say, it is: \begin{align*} \hat{p}&=\frac{y-F}{T-F} \end{align*}

where we defined $$y:=\frac{1}{N}\sum_i^N Y_i$$, which is the overall percentage of positive test results. The variance of $$\hat{p}$$ is

\begin{align*} Var(\hat{p})&=\frac{\sum_i^N Var(Y_i)}{N^2(T-F)^2}\\ &= \frac{Var(Y_i)}{N(T-F)^2}\\ &= \frac{\Big(F+(T-F)p\Big)\Big(1-F-(T-F)p\Big)}{N(T-F)^2} \end{align*}

The standard error is the square root of the above, while substituting in our estimate $$\hat{p}$$:

\begin{align*} SE(\hat{p})&=\sqrt{\frac{\Big(F+(T-F)\hat{p}\Big)\Big(1-F-(T-F)\hat{p}\Big)}{N(T-F)^2}}\\ &=\sqrt{\frac{y(1-y)}{N(T-F)^2}} \end{align*}

This expression is consistent with our intuition that the standard error is small (ie. high certainty in our estimation of the prevalence $$p$$) when $$N$$ is large, $$T$$ is large or $$F$$ is small.

Footnote: what if $$T=F$$? Intuitively, this means the test is worthless; sick and healthy patients are just as likely to test positive, meaning the test is completely uncorrelated to the disease process. Hence there's no point trying to estimate $$p$$ from such a test.

• Thanks, cambridgecircus for you clear and most helpful answer, which has helped me learn. Commented Jan 30 at 18:46
• The comments on the original post raise an interesting question. Suppose $T$ and $F$ as random variables with known probabilities, such as $T=p(Y=1|X=1)$, where $X$ is the patient's true state, disease or not. The $Var(Y_i)$ is the variance of the sum of two (negatively) correlated terms: $Tp$ and $F(1-p)$. The algebra simplifies nicely. However, can we treat $T$ and $F$ as random variables when addressing the variance of $Y_i$ but treat them as parameters elsewhere in the equation for $\hat{p}$? Commented Jan 30 at 19:00
• @GaryS no worries, if this answer has helped, please mark it as answered. Regarding your follow-on comment, yes it's possible to have $T$ and $F$ as RVs themselves. However I'm not too sure what yo mean by $Tp$ and $F(1-p)$ being negatively correlated, and the algebra simplyfing. Also I'm not sure what you mean by treating $T$,$F$ as RVs but parameters elsewhere; the point is that $\hat{p}$ will be a compound probability distribution parameterised by $T$ and $F$, which are RVs themselves. Commented Jan 31 at 14:08