# Trying to understand the derivation of expectation of sample mean $E(\bar x)= \mu$ where $\mu$ is the mean of the population

$$\bullet$$ Prove: $$E(\bar x)= \mu$$

# Answer:

Let $$x_1,x_2,x_3\ldots,x_n$$ denote the sample observations. The sample mean is $$\bar x= \frac{(x_1+x_2+x_3+\ldots+x_n)}{n}= \frac{1}{n}\sum x_i$$ where $$x_i$$ is the $$i$$-th member of of the sample.

Note, in simple random sampling(with or without replacement), the sample members has the same probability distribution as in the variable $$x$$ in the population.

Therefore, $$\mathrm E(\bar x_i)= \mu$$

And $$\mathrm E(\bar x)= \frac{1}{n}[\mathrm E(x_1)+ \mathrm E(x_2)+\ldots+\mathrm E(x_n)]= \mu.$$

What I'm not getting is the blocked part that the author wanted to highlight.

Why is $$E(\bar x_i)= n\,?$$

Can anyone tell me why actually $$x_i$$ has the same probability distribution as $$x$$ in the population especially even when the random sampling is done without replacement?

# Edit:

I've read this post. Here the derivation goes by

\begin{align}\mathrm E[x_i]&=\sum_{j=1}^N X_j\,\mathrm P[x_i=X_j]\\ &={1 \over N} \sum_{j=1}^N X_j\\ &={1 \over N} (N \bar{X})\\&= \bar{X}\;.\end{align}

But $$\mathrm P(x)\ne \frac{1}{N}$$ for sampling without replacement, isn't it?

Here $$\bar X = \mu\;.$$

What I'm saying is that $$x_i$$ can take any value from the population of size $$N$$ as prior to the choosing of $$i$$-th element, all the prior chosen element has been replaced back to the population and that's why the probability of choosing for $$x_i$$ remains the same viz. $$\frac{1}{N}\;;$$ but that is not the case for sampling without replacement for you can't have all the time the same population of size $$N$$.

Can anyone please help me understanding the case of sampling with replacement?

• $E(x_1)=E(x_2)=\dots=E(x_n)= \mu$, so you get $\frac{1}{n} ( \mu + \mu + \dots +\mu)$ as there are n terms in the sum, the latter is equal to $\frac{1}{n} n \mu$.
– user83346
Mar 20, 2016 at 11:11
• Please add the [self-study] tag & read its wiki. Mar 20, 2016 at 11:13
• @gung: Really? Is it required? I've been posting questions on Feynman and Purcell and many others on trying to understand what they want to mean by putting the excerpts at Physics; but didn't need such tag; however since I'm new, I would add this.
– user74724
Mar 20, 2016 at 11:25
• Yes, it is required. Thank you for adding the tag. Please read its wiki. Those are our policies. Also, please do not cross post, which is against SE policy. You should figure out which site you prefer and delete the other version. Mar 20, 2016 at 11:38
• It is unclear how the post you are referring to answers the same question? Can you edit the post explaining exactly what $x$, $\bar{x}$, $x_i$, $\bar{x}_i$ and $\mu$ are? The answer you are referring to is either wrong or does not apply to your problem. Mar 20, 2016 at 12:45

## 2 Answers

The marginal probability distribution of $$x_i$$ is the same as x in the population. Sampling without replacement causes the conditional distribution to differ. Ex: If you already know x1, then the probability distribution of x2 will be different than x in the population. However, one can ignore these dependencies when calculating E[$$\bar x$$]. E[X+Y]=E[X]+E[Y]. Therefore, one only need look at the marginal distribution of $$x_i$$, giving E[$$\bar x$$]=$$\mu$$*n/n

Random sampling, whether with or without replacement, gives the same distribution for each of the random samples!

This fact is very surprising to many people since they get hung up on the conditional distributions. Let's take a minimalistic example: an urn containing two balls numbered $$0$$ and $$1$$ respectively. We draw two balls from the urn and record their numbers. These numbers are our data $$x_1$$ and $$x_2$$. Our model is that the data are realizations of random variables $$X_1$$ and $$X_2$$, and in this example, we see that $$X_1$$ and $$X_2$$ can only take on values $$0$$ and $$1$$, and thus are Bernoulli random variables.

• Sampling with replacement:. We see that the possible outcomes of the experiment are $$(x_1, x_2) = (0, 0), \text{or}~ (0,1), \text{or}~(1,0), \text{or}~(1,1)$$ and all of these are equally likely to be observed. All this is consistent with $$X_1$$ and $$X_2$$ being Bernoulli random variables with parameter $$\frac 12$$ (that is, they are identically distributed), and furthermore, $$X_1$$ and $$X_2$$ are independent random variables.
• Sampling without replacement:.Now the possible outcomes of the experiment are $$(x_1, x_2) = (0,1), \text{or}~(1,0)$$ and both of these are equally likely to be observed. All this is consistent with $$X_1$$ and $$X_2$$ being Bernoulli random variables with parameter $$\frac 12$$ ( note that even on this case, $$X_1$$ and $$X_2$$ are identically distributed), but now $$X_1$$ and $$X_2$$ are very much dependent random variables: if you know the value of one of them, you know the value of the other.