Estimating the subset of a population which are sucessful when the population itself is an estimate

Let's say I have a population $$S$$, with an estimated size $$\hat{n}$$ (and standard error $$\sigma_{\hat{n}}$$). The way that $$\hat{n}$$ is estimated is through generating random samples from a larger sample space of size $$m$$ ($$n \ll m$$), and then determining how many belong to $$S$$. For our purposes, $$n$$ can't realistically be determined any other way. These samples form a Bernoulli distribution (since a sample either belongs to $$S$$ or doesn't), and we calculate $$\sigma_\hat{n}$$ through normal approximation.

I'd like to sample from $$S$$ and determine how many samples belong to $$T$$, on the basis of some arbitrary criteria for $$s \in S$$. Let the observed proportion of $$S$$ which are in $$T$$ be called $$\hat{p}$$, and let's say we also use a normal approximation. My question is: how does $$\sigma_\hat{n}$$ "interact" with $$\sigma_\hat{p}$$? (since we want to calculate $$\hat{n}\hat{p}$$)

Some notes:

• $$\hat{n}$$ and $$\hat{p}$$ are independent. There's no relationship between the two.
• Let's say we're initially sampling from $$R$$ (of known size $$m$$) to find $$\hat{n}$$. Why not instead determine directly how many $$r \in T$$? The reason is, verifying that some $$r$$ or $$s$$ is in $$T$$ is very complex (PSPACE-hard). The maximum number of samples I can realistically verify to be in $$T$$ is so small that $$m\hat{q}$$ (where $$\hat{q}$$ is the observed proportion of $$R$$ in $$T$$) would have confidence intervals much too large to mean anything useful. So instead, I can achieve a very confident estimate of $$\hat{n}$$, and then sample from $$S$$ instead.

Any guidance appreciated.

Potential answer: propagation of normally-distributed errors, in our case when multiplying some $$\sigma_1$$ by $$\sigma_2$$: notes

Other comments: I initially asked some pretty incomprehensible questions, and really shouldn't have been given the time. Thanks for everyone's precious time, especially BruceET and whuber's.

• Your question is a bit vague; are you sampling to estimate a proportion, the mean of some quantity, or the population size? A very accessible place to start learning about sampling is the Penn State Stat Online Course STAT506, Sampling Theory and Methods. Good luck! Commented Dec 27, 2020 at 19:30
• Why don't you know the actual sample size? What are you trying to find out? What do you mean by 'successful'? Commented Dec 27, 2020 at 21:13
• Hi Bruce, can you see the updated question? I don't know the sample size because it's not something I can easily calculate. I'm trying to find out the size of the subset of the population which is successful. And by successful, I mean that a member of the population passes the "success criteria". Commented Dec 27, 2020 at 21:27
• I guess this question will remain closed, but I guess in essence what I was asking is how do we multiply two confidence intervals: stats.stackexchange.com/questions/305382/… Commented Dec 27, 2020 at 22:09
• That last comment helped me understand what you are trying to ask, so I would like to suggest that you consider editing the question to include a similar remark. It would help even more to provide more information about how $n$ and $\sigma_{\hat n}$ are estimated as well as about how you are able to obtain samples. Abstractly it's a strange situation and the description at least suggests the possibility that $\hat n$ and $\hat p$ are not independent, which may be an important consideration.
– whuber
Commented Dec 28, 2020 at 13:45

An alternative approach is to sample from the large population R of size m>n untill you have some fixed number of successes (samples from T).

The sampling is done by testing whether a sample is S and if it is S then you test whether it is T/success. (So you do not need to do all the time the costly test to see if a sample is in T)

The number of samples that you need is negative Binomial distributed and based in that you can estimate a probability $$\hat{p}$$ for the fraction of T and S among in R and $$\hat{p}m$$ will be the estimate for the size.

• Ok interesting, thanks Sextus. Would you mind checking my logic here if you have time? I agree that checking $r \in S$ before $r \in T$ makes sense. In my case, I expect $\hat{p} \approx 10^{-3}$. The standard deviation of a binomial distribution using a normal approximation is $(\frac{p(1-p)}{N})^{1/2} \approx {(\frac{10^{-3}}{N})^{1/2}}$. I'd like to have $\sigma_{\hat{p}} \leq \frac{\hat{p}}{10}$, which means that approximately I must have at least $(\frac{10^{-3}}{N})^{1/2} = 10^{-4}$ (ignoring the 1.96 multiplier in the case of 95% CI). Commented Dec 29, 2020 at 0:20
• In which case $N = 10^5$, but I think the greatest $N$ I can have is $10^3$. Maybe I'll have to have a think about how I can reduce the size of $m$, thus increasing $\hat{p}$. But then again, maybe the error propagation approach would be easier Commented Dec 29, 2020 at 0:20
• Ah, sorry. If I can rule out most samples on the basis that $r \notin S$, such that the probability of $r \in T$ given $r \in S$ is approximately $1$, then I only need to do the harder verification of $r \in T$ for $\frac{10^5}{10^3}$... which is definitely possible :) Commented Dec 29, 2020 at 0:26

Suppose you are sampling college admission test scores from a large high school district. Traditionally, the district mean on this test has been 280 with a standard deviation of 25. Then an approximate 95% CI for this year's district mean would be of the form $$\bar X \pm 2(25)/\sqrt{n}.$$ If you want this year's CI to have margin of error of $$\pm 10,$$ then you have $$50/\sqrt{n} \approx 50/\sqrt{n} = 10.$$ So you need a sample size of $$n \approx 25.$$

Suppose you sample $$n = 25$$ observations at random from $$\mathsf{Norm}(\mu=280, \sigma=25),$$ to get data z as below. (Sampling and computations in R.)

set.seed(2020)
x = rnorm(25, 289, 25)
summary(x);  length(x);  sd(x)
Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
234.9   267.9   282.3   284.0   299.3   326.0
[1] 25         # sample size
[1] 22.52312   # sample SD


The t.test procedure in R, provides a 95% CI $$(274.7,293.3)$$ as part of its output, captured below using $-notation. The margin of error for this sample is about 9.3. [Margins of error will vary from sample to sample, depending on the sample standard deviation: for example, four additional samples of size $$n=25$$ gave margins of error 9.7, 12.9, 8.9, and 10.7.] t.test(x)$conf.int
[1] 274.6643 293.2585
attr(,"conf.level")
[1] 0.95

• Hi Bruce, apologies but I've updated the question. I didn't have the question clear in my head initially Commented Dec 27, 2020 at 21:04
• Sorry. Updated version does not appear. Maybe someone else will give this a try. Commented Dec 27, 2020 at 21:09
• Ah, maybe that's because it's closed Commented Dec 27, 2020 at 21:09