Is the mean of a specific sample of the population an unbiased estimator of the mean of the overall population?

Question

I'm just at the beginning of learning this stuff. I learnt that we can obtain what is called the 'expected mean' of the sample, which is basically the mean that we'd expect to get if we sampled the sample many times.

In the specific case below, we have one sample of an overall population. The book then assumes that this specific case's mean and standard deviation is exactly the same as actual mean of the population. My understanding is that the specific case's mean and standard deviation of a given sample are not necessarily the same as those of the statistic X bar, which is what we usually use as the unbiased estimator of the mean of the population. Even when we look at another unbiased estimator of the mean of the original population, which is E(X), we still end up with the fact that this doesn't necessarily mean that the expected mean of the original population is the same as the mean of the specific case of a given sample

So, WHY is the mean of a specific case sample being used as an unbiased estimator of the mean of the entire population? Someone please help clear my misconceptions :)

Answer 1

The mathematical definition of an unbiased estimator is: $E[u(X_1, X_2,\dots,X_n)]=\theta$.

In English, this formula means that the expected value of a statistic, generally given as $u(X_1, X_2], \dots, X_n)$, equals the parameter (of the population) value.

While a parameter has a single (probably unknown) value, a statistic has a distribution of values (which come from the fact that we could hypothetically calculate the statistic for many different random samples). As such, we can calculate an expected value for our distribution. Expected value is just the value that is most likely to occur (in the case of a Normal distribution it would be the peak, i.e., $\mu$, of the distribution).

In the example given in the question, there seems to be the assumption that the distribution of the data is Normal (because you are asked to provide unbiased estimates of mean and variance). So, we want to know if $\bar x$, the sample mean, and $S^2$, sample variance are unbiased. We can prove the first part straight forwardly, and I show it below. There is a proof for the second part, but it is a little more complicated. I encourage you to research unbiased estimators for $\hat \sigma^2$.

Our proof starts with the definition of an unbiased estimator above.

$\begin{align} E[u(X_1, X_2,\dots,X_n)] &= \theta\\ E[\bar X] &= E[\frac1n\sum_{i=1}^nX_i]\\ &=\frac1n \sum_{i=1}^nE(X_i)\\ &=\frac1n \sum_{i=1}^n \mu\\ &=\frac1n(n\mu)\\ &=\mu\\ E[\bar X] &= \mu \end{align}$

On the second line, we replace the generic function, $u(X_1,X_2, \dots, X_n)$, with our estimator, the sample mean ($\bar X$). On the following line we replace $\bar X$ with the formula for the sample mean. The 3rd line moves terms appropriately (following rules/properties of Expectation). By definition, we know that $E[X_i]=\mu$, so we replace it in the 4th line. What follows is simple math. We end up with the result that, indeed, our sample mean is an unbiased estimator of the population mean.

I hope this answer provides an approachable explanation for someone who is relatively new to these ideas.

Answer 2

The post is revised with the comments to the original answer taken into account hopefully providing a more clear account of what I was trying to state:

Whether the mean is unbiased is tightly coupled to the sampling design. Provided a simple random sample the sample mean is an unbiased estimator of the population parameter because over many samples the mean does not systematically overestimate or underestimate the true mean of the population.

A biased estimator of a population parameter will on the other hand systematically either over- or underestimate the population parameter.

With probability samples other than a simple random sample y for each element (i) must be weighted against the selection probability in order for the mean to be unbiased (look up for instance the Horvitz-Thompson estimator for a more detailed explanation).

Feel free to correct any incorrect or imprecise statements.

Answer 3

The sample mean is an unbiased estimator for the population mean. An estimator is a random variable with a probability distribution of its own. One specific sample with one specific value provides only one possible value of this (estimator) random variable.

Answer 4

This is how I think of this: expectation is the value you would expect to get when you sample a 'point' randomly from a population. So, if you approximate the mean using only one sample then, in expectation this should equal the population mean (because, as I said earlier, expectation is the value you expect to get when you sample from a population). Therefore, the mean is an unbiased estimator.

For a mathematical proof see this answer: https://stats.stackexchange.com/a/202669/218083