What percentage of a population needs a test in order to estimate prevalence of a disease? Say, COVID-19 A group of us got to discussing what percentage of a population needs to be tested for COVID-19 in order to estimate the true prevalence of the disease.  It got complicated, and we ended the night (over zoom) arguing about signal detection and characteristics of the imagined test.  I'm still thinking about it...
So:


*

*Assuming a perfect test, how do you plot the curve of testing reducing the confidence interval around true population infection rate?

*Assuming an imperfect test, how do you introduce the signal detection problem of test false positives and negatives?

*How do you model all this over time?
I'd love a textbook answer, a reference to a paper (ideally with math, not code), or a convincing argument.
 A: 1)  Making some assumptions about the population size (namely that it is large enough that a binomial model is appropriate), the prevalence of a disease in a population at a particular time can be obtained by sampling simple random sampling of people and finding who is sick.  That is a binomial random variable and the Wald confidence interval for a proportion $p$ is
$$ p \pm 1.96\dfrac{\sqrt{p(1-p)}}{\sqrt{n}}$$
The variance portion is bounded above by 0.5, so we can make the simplifying assumption that the width of the confidence interval is $\sim 2/\sqrt{n}$. So, the answer to this part is that the confidence interval for $p$ decreases like $1/\sqrt{n}$.  Quadruple your sample, halve your interval.  Now, this was based on using a Wald interval, which is known to be problematic when $p$ is near 0 or 1, but the spirit remains the same for other intervals.
2) You need to look at metrics like specificity and sensitivity.  
Sensitivity is the probability that a diseased person will be identified as diseased (i.e. tests positive).  Specificity is the probability that a person without the disease is identified as not having the disease (i.e. tests negative).  There are lots of other metrics for diagnostic tests found here which should answer your question.
3) I guess this is still up in the air.  There are several attempts to model the infection over time.  SIR models and their variants can make a simplifying assumption that the population is closed (i.e. S(t) + I(t) + R(t) = 1) and then I(t) can be interpreted as the prevalence.  This isn't a very good assumption IMO because clearly the population is not closed (people die from the disease).  As for modelling the diagnostic properties of a test, those are also a function of the prevalence.  From Bayes rule
$$ p(T+ \vert D+) = \dfrac{P(D+\vert T+)p(T+)}{p(D+)}$$
Here, $P(D+)$ is the prevalence of the disease, so as this changes then the sensitivity should change as well.
A: It has been answered by Dimitri Pananos, I will only add that in order to estimate the prevalence with pre-set precision you need an absolute sample size which is pretty much invariant with the population size (only when the sample is a substantial part of the target population you have a non-negligible finite population correction factor). So there is not a percentage of the population that needs to be tested: 50% of a small population may not be sufficient, 0.5% of a large population may be far enough for the same precision.
A: I'll go in a somewhat different direction and say that it depends...


*

*Of course any sampling is based on the notion that the sampling is truly random. Trying to account for non-randomness in the sample tremendously complicates the situation. 

*This type of yes/no measurement is non-parametric. Such tests need a larger sample size than if the measurement was parametric. 

*Presumably you're ignoring the problem of false positives and false negatives in the testing. False positives could be a real problem is the disease fraction is low. 

*What is the actual fraction of the diseased? If only 0.1% of the population is diseased then on average one 1 in a 1000 tests would be positive. So the lower the infection rate, the larger the sample would need to be. 

*How precise an estimate do you want? In other words do you want to know the infection rate +/- 20%, or to say +/- 1%. The more precise you want to know the value of the infection rate the larger the sample would need to be. 
There is a type of statistical testing called Acceptance Testing which can be used. Basically the important decision is how precise do you want the measurement to be? Then you keep sampling until that level of precision is achieved. So if 50% of the population is infected then a relatively small sample is needed to get to +/- 10% error in the measurement itself (e.g 50% +/- 5%). However if only 0.5% of the population is infected then a much larger sample is needed to determine that the disease level (e.g.  0.5% +/- 0.05%).  
