# Does sampling from a larger sample change how much variance around an estimate to expect?

Suppose you have one billion marbles. Each marble is either red or white. Call the proportion that are red, p. You randomly sample 100 marbles. Based on this sample you estimate $\hat{p}$. You can construct a 95% confidence interval for $\hat{p}$, if I have this right, like this:

$$\hat{p} \pm 1.96\sqrt{\frac{1}{100}\hat{p}(1-\hat{p})}$$

Now, here is my question. Suppose that instead of randomly drawing 100 marbles directly from the population, you first randomly draw 1000 marbles, and then randomly draw 100 marbles from that sample to take your estimate. Or suppose further that you first randomly draw 10,000 marbles, then randomly draw 1,000 marbles from that 10,000, and finally randomly draw 100 marbles from that 1,000 and use the 100 to make your estimate. How does the confidence interval associated with your estimate change? If at all?

My first thought was that it doesn't matter at all, but I'm not totally sure if that's right. To give some added context, I am a biologist, and the marbles are cells, and the repeated sampling is serial dilutions prior to the final assay of 100 cells for some (binary) trait.

• Are they all simple random samples? Is the sampling procedure unbiased? – gung - Reinstate Monica Aug 28 '15 at 19:34
• Yes. Will edit to make clear. – Ben S. Aug 28 '15 at 20:04

Your intuition is correct, provided that the sampling at each phase is simple random sampling without replacement. This is not clear from your description. It would be false if, for example, cells were selected in clusters.

If there is simple random sampling at each phase, then the 100 marbles that result from the three-phase sampling process you describe are, in fact, a simple random sample of size 100 from the original population of 1 billion. (a proof is below.) Therefore the equation for the confidence interval is correct.

If cells are not selected as simple random samples at each phase, you will have to provide more details. I'll just note that some selection schemes can provide a close equivalent to simple random sampling without replacement. This can happen, for example, in drawing lottery tickets (or marbles) if there is sufficient mixing and if there are no physical differences between the tickets.

In what follows, all sampling is simple random sampling without replacement (WOR).

Informal proof: Let $p_1 =\dfrac{n_1}{N}$ be the probability of selection in the first sample, and let $p_2$ be the conditional probability of selection in the sample: $p_2 =\dfrac{n_2}{n_1}$. The overall probability of selection to the second sample is:

\begin{align} f & = \frac{n_1}{N}\times\frac{n_2}{n_1} \\ & = \frac{n_2}{N} \end{align} This is the same probability that would obtain if you initially took a simple random sample of size $n_2$.

Formal proof: The proof above only shows that the probability of selecting a marble for the final sample of size $n_2$ in two phases is the same as the probability if a simple random sample of $n_2$ without replacement had been drawn from the original population. This is not quite the same as saying that the result of the two-phase process is a simple random sample WOR of size $n_2$ from the original population.

Recall that the number of distinct simple random samples of size $n$ that can be drawn without replacement (WOR) from population of size $N$ is:

$${N \choose n} = \frac{N!}{n!(N-n)!}$$ (Cochran, 1977, page 18).

Let $S_2$ designate one distinct sample of size $n_2$. The probability of drawing $S_2$ in a simple random sample WOR of size $n_2$ is

$$f_{S_2} = \frac{1}{{N \choose n_2}}=\frac{N!}{n_2!(N-n_2)!}$$

i.e. it is the probability of drawing any sample of size $n_2$ from $N$ observations.

Alternatively, we will select $S_2$ in two phases: a) Draw a sample of size $n_1$ that contains $S_2$ from the $N$ elements of the population. b) From this initial sample, draw a sample of size $n_2$ which turns out to be $S_2$. We will denote the probabilities of these two events as $f_a$ and $f_b$, respectively. We compute $f_a$ and $f_b$ and leave to you the simple calculation that shows:

$$f_{S_2} = f_a \times f_b$$

Phase a: What is the number SRSs of size $n_1$ drawn WOR that will contain $S_2$? It is:

$$K = { N-n_2 \choose n_1 - n_2} =\frac{(N-n_2)!}{(N-n_1)!(n_1-n_2)!}$$

(The logic is simple: remove $S_2$ from the population, leaving $N-n_2$ observations. Then draw $n_1-n_2$ of them and add back $S_2$ to complete the sample of size $n_1$.)

The probability of drawing such a sample is found by dividing $K$ by the total number of samples of size $n_1$ that can be drawn from $N$, yielding: $$f_a =\frac{K}{{N \choose n_1}} = \frac{(N-n_2)! n_1!}{(n_1-n_2)! N!}$$

Phase b: Given that a sample of size $n_1$, containing $S_2$, has been drawn, what is the probability that a second random sample of size $n_2$ will be exactly $S_2$?

The number of samples of size $n_2$ that can be drawn from $n_1$ is:

$${n_1 \choose n_2}= \frac{n_1!}{(n_1 - n_2)! n_2!}$$ and exactly one of them is $S_2$. So the probability of drawing $S_2$ in this phase, conditional on the event that $S_2$ is contained in the first phase sample of size $n_1$ is:

$$f_b = \frac{1}{{n_1 \choose n_2}}= \frac{(n_1 - n_2)! n_2!}{n_1!}$$ If the result of this is a simple random sample of size $n_2$ drawn from $N$, then it must be true that:

$$f_{S_2} = f_a \times f_b$$ and this is easily shown.

To sum up: A simple random sample WOR of a simple random sample WOR is itself a simple random sample WOR of the original population.*

Reference: WG Cochran (1977) Sampling Techniques, Wiley, New York.

• This is exactly what I meant (sampling WOR). Thanks. – Ben S. Aug 31 '15 at 1:49