1
$\begingroup$

I'm just at the beginning of learning this stuff. I learnt that we can obtain what is called the 'expected mean' of the sample, which is basically the mean that we'd expect to get if we sampled the sample many times.

In the specific case below, we have one sample of an overall population. The book then assumes that this specific case's mean and standard deviation is exactly the same as actual mean of the population. My understanding is that the specific case's mean and standard deviation of a given sample are not necessarily the same as those of the statistic X bar, which is what we usually use as the unbiased estimator of the mean of the population. Even when we look at another unbiased estimator of the mean of the original population, which is E(X), we still end up with the fact that this doesn't necessarily mean that the expected mean of the original population is the same as the mean of the specific case of a given sample

So, WHY is the mean of a specific case sample being used as an unbiased estimator of the mean of the entire population? Someone please help clear my misconceptions :)

enter image description here

enter image description here

$\endgroup$
2
$\begingroup$

The post is revised with the comments to the original answer taken into account hopefully providing a more clear account of what I was trying to state:

Whether the mean is unbiased is tightly coupled to the sampling design. Provided a simple random sample the sample mean is an unbiased estimator of the population parameter because over many samples the mean does not systematically overestimate or underestimate the true mean of the population.

A biased estimator of a population parameter will on the other hand systematically either over- or underestimate the population parameter.

With probability samples other than a simple random sample y for each element (i) must be weighted against the selection probability in order for the mean to be unbiased (look up for instance the Horvitz-Thompson estimator for a more detailed explanation).

Feel free to correct any incorrect or imprecise statements.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Unbiasedness is a property of estimators(i.e formulas) not actual estimates/numbers. So don't include confidence intervals in your answer. Also this does not answer the OP's question>> $\endgroup$ – machazthegamer May 5 '17 at 12:12
  • $\begingroup$ Original question has problem. "Calculate unbiased estimates of ..." should be "Use unbiased estimator to estimate ..." $\endgroup$ – user158565 May 5 '17 at 15:12
  • $\begingroup$ It's likely worthwhile to maintain the conventional distinction between a "simple random sample"--in which all elements have equal chances of being chosen and those choices are independent--and a "probability sample," which is a far more general concept in which elements may have difference chances of selection. It would also be more accurate to retain the logical distinction between "only under" (meaning "if and only if") and "under" (meaning "is implied by"), because the sample mean can be unbiased for many other kinds of sampling procedures. $\endgroup$ – whuber May 5 '17 at 15:33
  • $\begingroup$ @machazthegamer indeed. I will remove that section for the purpose of clarity and relevance to the OP's question. $\endgroup$ – MiGH May 6 '17 at 5:42
  • 1
    $\begingroup$ @whuber good point. Based on your good input I have revised my answer. $\endgroup$ – MiGH May 6 '17 at 5:46
2
$\begingroup$

The mathematical definition of an unbiased estimator is: $E[u(X_1, X_2,\dots,X_n)]=\theta$.

In English, this formula means that the expected value of a statistic, generally given as $u(X_1, X_2], \dots, X_n)$, equals the parameter (of the population) value.

While a parameter has a single (probably unknown) value, a statistic has a distribution of values (which come from the fact that we could hypothetically calculate the statistic for many different random samples). As such, we can calculate an expected value for our distribution. Expected value is just the value that is most likely to occur (in the case of a Normal distribution it would be the peak, i.e., $\mu$, of the distribution).

In the example given in the question, there seems to be the assumption that the distribution of the data is Normal (because you are asked to provide unbiased estimates of mean and variance). So, we want to know if $\bar x$, the sample mean, and $S^2$, sample variance are unbiased. We can prove the first part straight forwardly, and I show it below. There is a proof for the second part, but it is a little more complicated. I encourage you to research unbiased estimators for $\hat \sigma^2$.

Our proof starts with the definition of an unbiased estimator above.

$\begin{align} E[u(X_1, X_2,\dots,X_n)] &= \theta\\ E[\bar X] &= E[\frac1n\sum_{i=1}^nX_i]\\ &=\frac1n \sum_{i=1}^nE(X_i)\\ &=\frac1n \sum_{i=1}^n \mu\\ &=\frac1n(n\mu)\\ &=\mu\\ E[\bar X] &= \mu \end{align}$

On the second line, we replace the generic function, $u(X_1,X_2, \dots, X_n)$, with our estimator, the sample mean ($\bar X$). On the following line we replace $\bar X$ with the formula for the sample mean. The 3rd line moves terms appropriately (following rules/properties of Expectation). By definition, we know that $E[X_i]=\mu$, so we replace it in the 4th line. What follows is simple math. We end up with the result that, indeed, our sample mean is an unbiased estimator of the population mean.

I hope this answer provides an approachable explanation for someone who is relatively new to these ideas.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ +1 Very nicely explained. As was pointed out to me on this site many years ago, though, none of the results here require a Normal distribution. $\endgroup$ – whuber Mar 20 at 14:03
0
$\begingroup$

The sample mean is an unbiased estimator for the population mean. An estimator is a random variable with a probability distribution of its own. One specific sample with one specific value provides only one possible value of this (estimator) random variable.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ These assertions merely restate what's in the question. What the OP would like is an explanation of why the sample mean is unbiased. $\endgroup$ – whuber May 5 '17 at 15:34
0
$\begingroup$

This is how I think of this: expectation is the value you would expect to get when you sample a 'point' randomly from a population. So, if you approximate the mean using only one sample then, in expectation this should equal the population mean (because, as I said earlier, expectation is the value you expect to get when you sample from a population). Therefore, the mean is an unbiased estimator.

For a mathematical proof see this answer: https://stats.stackexchange.com/a/202669/218083

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.