Expected value of iid random variables

Question

I came across this derivation which I don't understand: If $X_1, X_2, ..., X_n$ are random samples of size n taken from a population of mean $\mu$ and variance $\sigma^2$, then

$\bar{X} = (X_1 + X_2 + ... + X_n)/n$

$E(\bar{X}) = E(X_1 + X_2 + ... + X_n)/n = (1/n)(E(X_1) + E(X_2) + ... + E(X_n))$

$E(\bar{X}) = (1/n)(\mu + \mu + ...n ~\text{times}) = \mu$

This is where I'm lost. The argument used is $E(X_i) = \mu$ because they are identically distributed. In reality this isn't true. Suppose I have a sample, $S=\{1,2,3,4,5,6\}$ and then if randomly select 2 numbers with replacement and repeat this procedure 10 times, then I get 10 samples: (5, 4) (2, 5) (1, 2) (4, 1) (4, 6) (2, 4) (6, 1) (2, 4) (3, 1) (5, 1). This is how it looks like for 2 random variables $X_1, X_2$. Now if I take the expectation value of $X_1$ I get,

$E(X_1) = 1.(1/10) + 2.(3/10) + 3.(1/10) + 4.(2/10) + 5.(2/10) + 6.(1/10) = 34/10 = 3.4$

But the expected value of the population is 3.5. What is actually wrong in my reasoning?

Answer 1

First of all, $X_1, X_2,...,X_n$ are not samples. These are random variables as pointed out by Tim. Suppose you are doing an experiment in which you estimate the amount of water in a food item; for that you take say 100 measurements of water content for 100 different food items. Each time you get a value of water content. Here the water content is random variable and Now suppose there were in total 1000 food items which exist in the world. 100 different food items will be called a sample of these 1000 food items. Notice that water content is the random variable and 100 values of water content obtained make a sample.

Suppose you randomly sample out n values from a probability distribution, independently and identically, It is given that the $E(X)=\mu$. Now you need to find out expected value of $\bar{X}$. Since each of $X_i$ is independently and identically sampled, expected value of each of the $X_i$ is $\mu$. Therefore you get $\frac{n\mu}{n} =\mu$.

The third equation in your question is the condition for an estimator to be unbiased estimator of the population parameter. The condition for an estimator to be unbiased is

$$ E(\bar{\theta})=\theta $$

where theta is the population parameter and $\bar{\theta}$ is the parameter estimated by sample.

In your example you population is $\{1,2,3,4,5,6\}$ and you have been given a sample of $10$ i.i.d. values which are $\{5,2,1,4,4,2,6,2,3,5\}$. The question is how would you estimate the population mean given this sample. According to above formula the average of the sample is an unbiased estimator of the population mean. The unbiased estimator doesn't need to be equal to actual mean, but it is as close to mean as you can get given this information.