3
$\begingroup$

Notation

$\mathcal{X}$ is the multiset of size $N$ of all possible values in our population. These values are fixed.

$\mathcal{S}$ is the set of all $\binom{N}{n}$ possible subsets of $\mathcal{X}$ of fixed size $n$.

Let $S\in\mathcal{S}$ be a random variable representing a simple random sample from $\mathcal{X}$ without replacement.

$x_i \in \mathcal{X}$ is the $i^{th}$ member of the population, $i\in\{1,...,N\}$.

$x_{si}$ is the $i^{th}$ member of a given sample $s\in\mathcal{S}$, $i\in\{1,...n\}$

Claim

$\frac{1}{n}\sum_{i=1}^{n} x_{Si}$ is an unbiased estimate of $\frac{1}{N}\sum_{i=1}^{N} x_i$

Proof (Cochran 1977)

Since every unit $x_i$ appears in the same number of samples, it is clear that:

$\mathbb{E} \sum_{i=1}^n x_{Si}$ must be some multiple of $\sum_{i=1}^{N} x_i$

The multiplier must be $n/N$, since the expression on the left has $n$ terms and that on the right has $N$ terms. This leads to the result.

Question

I'm familiar with the other proofs of the unbiasedness of this estimator using indicator variables, linearity of expectation, and combinatorics, but I'm having a hard time wrapping my head around the argument being made here.

The argument that every unit appears in the same number of samples is clear to me. Everything else that follows is not. In particular:

  • Why does the left-hand expression have to be a multiple of the right?
  • How do we get the multiplier from the number of terms?
$\endgroup$
  • $\begingroup$ Unless $\sum_{i=1}^N x_i=0,$ "must be some multiple of" is without content, because every real number is "some multiple of" any nonzero real number. The multiple, however, does not depend on the $x_i$: it depends only on $n$ and $N$. This follows upon observing the whole setup is invariant under all permutations of the $x_i$ and both sides are linear functions of $(x_1, x_2, \ldots, x_N).$ $\endgroup$ – whuber Mar 14 '18 at 20:03
  • $\begingroup$ @whuber I can see the invariance under permutations of $x_i$ and that both sides are linear functions of $(x_1, x_2, ..., x_N)$, but I don't see how that buys us the non-dependence on $x_i$ and dependence solely on $n$ and $N$. Would you mind expanding the logic a bit? $\endgroup$ – giftbox Mar 14 '18 at 22:58
2
$\begingroup$

The Cochran proof is a slicker variant of the indicator proof. (It might be the same one you're familiar with.) If $S$ is a randomly selected subset with $n$ elements, then the following holds as an identity between random variables: $$ \sum_{i=1}^n x_{{\cal S}i} = \sum_{j=1}^N x_jI(x_j\in S)\tag 1 $$ Since all subsets of $n$ elements are equally likely, the expectation of (1) is then $$ E\sum_{i=1}^n x_{{\cal S}i} \stackrel{(1)}= \frac1{|{\cal S}|}\sum_{X\in{\cal S}}\left[\sum_{j=1}^N x_jI(x_j\in X)\right] \stackrel{(2)}= \sum_{j=1}^N x_j\left[\frac1{|{\cal S}|}\sum_{X\in{\cal S}}I(x_j\in X)\right] \stackrel{(3)}= \sum_{j=1}^N x_jP(x_j\in S) $$ where $\sum_{X\in \cal S}$ denotes summation over every subset (nonrandom) of $n$ elements; there are $|\cal S|$ possible values for $X$. Step (2) is an interchange of summation. In step (3) we realize that each $x_j$ is seen in the same number of subsets, so all the $P(x_j\in S)$ are the same, and equal to $n/N$ since that's the probability that a given element from a set of $N$ items will be chosen to appear in a subset of $n$.

$\endgroup$
  • $\begingroup$ This is indeed the indicator proof I'm familiar with. I'm curious, however, of how the Cochran proof skips straight to the end of that chain of logic. It doesn't even mention a probability $P(x_j \in S)$, which is where I naturally see the $n/N$ coming in. $\endgroup$ – giftbox Mar 14 '18 at 22:51
  • $\begingroup$ @giftbox I agree it's not obvious where $n/N$ arises -- it's not clear why it's enough to say "the left has $n$ terms and the right has $N$ terms". By avoiding summations Cochran is trying to reveal the simple concept at the heart of the indicator proof but in doing so he renders the argument a bit too terse. $\endgroup$ – grand_chat Mar 14 '18 at 23:08
  • $\begingroup$ @giftbox I just noticed that $E\sum_{i=1}^n x_{{\cal S}i} = \sum_{j=1}^N x_jP(x_j\in S)$ follows immediately from (1), rendering the indicator proof even more slick :) $\endgroup$ – grand_chat Mar 14 '18 at 23:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.