Trying to understand the derivation of expectation of sample mean $E(\bar x)= \mu$ where $\mu$ is the mean of the population

Question

$\bullet$ Prove: $E(\bar x)= \mu$

Answer:

Let $x_1,x_2,x_3\ldots,x_n$ denote the sample observations. The sample mean is $$\bar x= \frac{(x_1+x_2+x_3+\ldots+x_n)}{n}= \frac{1}{n}\sum x_i$$ where $x_i$ is the $i$-th member of of the sample.

Note, in simple random sampling(with or without replacement), the sample members has the same probability distribution as in the variable $x$ in the population.

Therefore, $\mathrm E(\bar x_i)= \mu$

And $$\mathrm E(\bar x)= \frac{1}{n}[\mathrm E(x_1)+ \mathrm E(x_2)+\ldots+\mathrm E(x_n)]= \mu.$$

---

What I'm not getting is the blocked part that the author wanted to highlight.

Why is $E(\bar x_i)= n\,?$

Can anyone tell me why actually $x_i$ has the same probability distribution as $x$ in the population especially even when the random sampling is done without replacement?

Edit:

I've read this post. Here the derivation goes by

\begin{align}\mathrm E[x_i]&=\sum_{j=1}^N X_j\,\mathrm P[x_i=X_j]\\ &={1 \over N} \sum_{j=1}^N X_j\\ &={1 \over N} (N \bar{X})\\&= \bar{X}\;.\end{align}

But $\mathrm P(x)\ne \frac{1}{N} $ for sampling without replacement, isn't it?

Here $\bar X = \mu\;.$

What I'm saying is that $x_i$ can take any value from the population of size $N$ as prior to the choosing of $i$-th element, all the prior chosen element has been replaced back to the population and that's why the probability of choosing for $x_i$ remains the same viz. $\frac{1}{N}\;;$ but that is not the case for sampling without replacement for you can't have all the time the same population of size $N$.

Can anyone please help me understanding the case of sampling with replacement?

---

Cross-posted: Trying to understand the derivation of expectation of sample mean $E(\bar x)= \mu$ where $\mu$ is the mean of the population

Answer 1

The marginal probability distribution of $x_i$ is the same as x in the population. Sampling without replacement causes the conditional distribution to differ. Ex: If you already know x1, then the probability distribution of x2 will be different than x in the population. However, one can ignore these dependencies when calculating E[$\bar x$]. E[X+Y]=E[X]+E[Y]. Therefore, one only need look at the marginal distribution of $x_i$, giving E[$\bar x$]=$\mu$*n/n

Answer 2

Random sampling, whether with or without replacement, gives the same distribution for each of the random samples!

This fact is very surprising to many people since they get hung up on the conditional distributions. Let's take a minimalistic example: an urn containing two balls numbered $0$ and $1$ respectively. We draw two balls from the urn and record their numbers. These numbers are our data $x_1$ and $x_2$. Our model is that the data are realizations of random variables $X_1$ and $X_2$, and in this example, we see that $X_1$ and $X_2$ can only take on values $0$ and $1$, and thus are Bernoulli random variables.

• Sampling with replacement:. We see that the possible outcomes of the experiment are $$(x_1, x_2) = (0, 0), \text{or}~ (0,1), \text{or}~(1,0), \text{or}~(1,1)$$ and all of these are equally likely to be observed. All this is consistent with $X_1$ and $X_2$ being Bernoulli random variables with parameter $\frac 12$ (that is, they are identically distributed), and furthermore, $X_1$ and $X_2$ are independent random variables.
• Sampling without replacement:.Now the possible outcomes of the experiment are $$(x_1, x_2) = (0,1), \text{or}~(1,0)$$ and both of these are equally likely to be observed. All this is consistent with $X_1$ and $X_2$ being Bernoulli random variables with parameter $\frac 12$ ( note that even on this case, $X_1$ and $X_2$ are identically distributed), but now $X_1$ and $X_2$ are very much dependent random variables: if you know the value of one of them, you know the value of the other.