I was reading about the proof of the sample mean being the unbiased estimator of population mean. Here is the concerned derivation:

Let us consider the simple arithmetic mean $\bar y = \frac{1}{n}\,\sum_{i=1}^{n} y_i$ as an unbiased estimator of population mean $\overline Y = \frac{1}{N}\,\sum_{i=1}^{N} Y_i$.

Simple Random Sampling Without Replacement

Let $t_i = \sum_{i=1}^n y_i\;.$

\begin{align}\mathrm E(\bar y)&= \frac{1}{n}\,\mathrm E{\left(\sum_{i=1}^n y_i\right)}\\&= \frac{1}{n}\,\mathrm E(t_i)\\ &= \frac{1}{n}\color{red}{\left(\frac{1}{N \choose n}\,\sum_{i=1}^{N \choose n}t_i\right)}\\ &= \frac{1}{n}\left(\frac{1}{N \choose n}\,\sum_{i=1}^{N \choose n}\left(\sum y_i\right)\right)\\ &= \frac{1}{n}\left(\frac{1}{N \choose n}\,\color{red}{{N-1\choose n-1}\sum_{i=1}^N y_i}\right)\\ &=\frac{1}{N}\sum_{i=1}^{N}y_i\\ &= \overline Y\;.\end{align}

I couldn't understand the derivation; I really couldn't conceive how the red coloured terms came from nowhere.

Can anyone please help me explain the derivation by showing how the red coloured terms came in the concerned steps?


The author's derivation is pretty strange. The fact that $\bar{y}$ is an unbiased estimate of $\mu$ the population mean when sampling without replacement is true due to linearity of expectation alone:

\begin{align} \text{E}(\bar{y}) &= \text{E} \left ( \frac{1}{n} \sum_{i=1}^{n} y_i \right ) \\ &= \frac{1}{n} \sum_{i=1}^{n} \text{E}(y_i) \\ &= \frac{1}{n} \sum_{i=1}^{n} \mu \\ &= \mu . \end{align}

There's no reason to bother with combinatorics.

  • $\begingroup$ Is your applicable for sampling without replacement? $\endgroup$
    – user74724
    Mar 20 '16 at 17:31
  • $\begingroup$ Yes. Because expectation is linear the lack of independence between observations doesn't matter. $\endgroup$
    – dsaxton
    Mar 20 '16 at 17:33
  • $\begingroup$ Can you help me in this question of mine? I'd be grateful. $\endgroup$
    – user74724
    Mar 20 '16 at 17:36
  • 1
    $\begingroup$ I don't fully understand the notation you've used in that question, but it seems to be the same issue as here. The idea in both questions is that when taking an expectation of a sum, it does not matter if the terms are dependent upon one another, the expectations will add. $\endgroup$
    – dsaxton
    Mar 20 '16 at 18:08
  • 1
    $\begingroup$ If you draw $N - 1$ samples, set them aside without looking at them, and then draw the $N^\text{th}$, what do you expect it to equal? $\endgroup$
    – dsaxton
    Mar 21 '16 at 13:22

Ok this question has been silent for a while, but I was intrigued and researched it. Here is what I came up with:

The term in red on the third line arises as follows:

We can calculate the expectation for a discrete random variable, $X$ using $$\text{E}(X) = \sum_{i=1}^k x_i p_i$$ where we have a finite number of outcomes, $x_1,x_2,\dots,x_k$ that occur with probabilities $p_1, p_2, \dots, p_k$ respectively.

In the case of SRSWOR, we are selecting $n$ (sample size) units out of $N$ (population). The total number of possible samples is $N \choose n$, since the probability of choosing a sample is uniform, the probability of selecting any one of these samples is $\frac{1}{N \choose n}$.

In this example, $x_i=t_i=\sum_{j=1}^n y_i$ and $k = {N \choose n}$ and $p_i = \frac{1}{N \choose n}$. So, we have \begin{align} E[t_i] &= \sum_{i=1}^{N \choose n} \frac{1}{N \choose n} t_i\\ &= \frac{1}{N \choose n} \sum_{i=1}^{N \choose n} \sum_{i=1}^n y_i\\ \end{align}

The term in red on the 5th line, I can only provide a high level intuition for. The short answer is essentially, $$\sum_{i=1}^{N \choose n} \sum_{i=1}^n y_i = {N-1 \choose n-1} \sum_{i=1}^N y_i$$ Note on the right that the summation now goes from 1 to $N$ instead of 1 to $n$.

You do some straightforward (?) math to find $$ \frac1n \frac{1}{N \choose n}{N-1 \choose n-1} = \frac1N$$ And you have your proof. Using combinatorics provides one way to gain intuition regarding key aspects of choosing n samples from a population of N possible samples without replacement (SRSWOR). In this case it gives us a way to determine whether the sample arithmetic mean is an unbiased estimator of the population mean at a deeper level.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy