explanation of proof that sample mean is unbiased

Question

William Cochran's book on sampling gives the following proof that a sample mean is unbiased:

Since every unit appears in the same number of samples, it is clear that $E[Y_1 + \cdots + Y_n]$ must be some multiple of $y_1+y_2+\cdots+y_N$. The multiplier must be $n/N$ since the first expression has $n$ terms and the second has $N$ terms.

I am trying to understand the last sentence, about the number of terms. The expectation here is taken over all samples of size $n$ (all of equal probability) out of a population of size $N$.

Answer 1

Let $y_1,\dots,y_N$ denote the values of some attribute of the population units $1,\dots,N$. In randomization theory, the $y_i$'s are not random variables, but fixed quantities. A random sample is determined by (dependent) indicator random variables $Z_1,\dots,Z_N\in\{0,1\}$ such that $Z_1+\dots+Z_N$ is equal to the sample size $n$. Each $Z_i$ determines if the respective population unit is included ($Z_i=1$) or not ($Z_i=0$) in the sample. The sample mean is the random variable $$ \bar{Y} = \frac{1}{n}\sum_{i=1}^N y_i Z_i. $$ For a simple random sample, if the $i$-th population unit is included in the sample, then the other $n-1$ sample units must be chosen from the remaining $N-1$ population units. Hence, the probability $\Pr\{Z_i=1\}$ is equal to the number of samples of size $n$ which include $i$, given by $n-1\choose N-1$, divided by the number of size $n$ samples, given by $n\choose N$: $$ \Pr\{Z_i=1\} = \frac{n-1\choose N-1}{n\choose N} = \frac{n}{N}. $$ It follows that $$ \mathrm{E}[\bar{Y}] = \frac{1}{n}\sum_{i=1}^N y_i \mathrm{E}[Z_i] = \frac{1}{N}\sum_{i=1}^N y_i = \bar{y}. $$ Therefore, the sample mean $\bar{Y}$ is an unbiased estimator of the population mean $\bar{y}$.

Answer 2

"The number $A$ must be some multiple of the number $B$." is true because the number $A/B$ exists, unless $B=0$. So Cochran's meaning may take some effort to discern.

Suppose $N=5$ and $n=3$. Then the possible samples (assuming it's without replacement) are these: \begin{align} & y_1 + y_2 + y_3 \\[3pt] & y_1 + y_2 + y_4 \\[3pt] & y_1 + y_2 + y_5 \\[3pt] & y_1 + y_3 + y_4 \\[3pt] & y_1 + y_3 + y_5 \\[3pt] & y_1 + y_4 + y_5 \\[3pt] & y_2 + y_3 + y_4 \\[3pt] & y_2 + y_3 + y_5 \\[3pt] & y_2 + y_4 + y_5 \\[3pt] & y_3 + y_4 + y_5 \end{align} There are $10$ of these, so the average value of the sum is the average of these $10$ sums. The number $y_1$ appears in six of these sums, and $y_2$ appears in six of them, and so one. Thus "every unit appears in the same number of samples", since in this case, that "same number" is $6$. Thus the sum of the ten numbers is $6y_1+6y_2+6y_3+6y_4+6y_5.$ Dividing by $10$ gives the average, or expected value, of the random sample of size $3$. Hence \begin{gather} \text{expected value of the sample sum} = \frac{6y_1+6y_2+6y_3+6y_4+6y_5}{10} \\[10pt] = \frac 6 {10} (y_1+y_2+y_3+y_4+y_5) = \frac 3 5 (y_1+y_2+y_3+y_4+y_5) \end{gather} This ratio, $\dfrac 3 5,$ is the ratio of the sample size to the population size. That is the "multiplier". I expect there is a short elegant combinatorial argument showing that that will happen in general, i.e. with other numbers than $3$ and $5$, and also making Cochran's argument precise, but I'm not sure how best to express that right now.

Answer 3

1. Every unit appears in the same number $k$ of samples

2. If $M$ is the total number of samples, $\Rightarrow M=k \cdot N $

3. \begin{align}\Rightarrow {} & E\left(\frac{y_1+\cdots+y_n}n\right) \\[8pt] = {} & \left(\bar{y}_1 + \cdots + \bar{y}_M\right)/M \\[8pt] = {} & (k \cdot Y_1+\cdots+ k \cdot Y_N)/M \\[8pt] = {} & \left. \left(\frac{M}{N} \cdot Y_1+\cdots+\frac{M}{N} \cdot Y_N \right)\right/M \\[8pt] = {} & \frac{1}{N} \cdot (Y_1+\cdots+ Y_N) \end{align}