5
$\begingroup$

Given a random variable $x$ with a well-defined expected value $\mu$, is the mean of the set of samples $\{x_1,\ \cdots,\ x_n\}$, which we'll call $\widehat{\mu}$, always an unbiased estimator of $\mu$? In other words, is it always true that: $$ E[\widehat{\mu}] = E[\frac{\sum_{i=1}^n x_i}{n}] = \mu = E[x]. $$ regardless of the specifics of $x$?

Further, would $\widehat{\mu}$ always be consistent, in the sense that the variance $V[\widehat{\mu}]$ would tend to decrease as the sample count $n$ increased?

$\endgroup$
  • 3
    $\begingroup$ The first question is answered immediately using the linarity of expectation. The second conclusion is true only when the underlying distribution has finite variance, in which case it follows with a simple computation of the variance. $\endgroup$ – whuber Jan 19 '18 at 19:53
5
$\begingroup$

Answered in comments: The first question is answered immediately using the linearity of expectation. The second conclusion is true only when the underlying distribution has finite variance, in which case it follows with a simple computation of the variance. – whuber

The second conclusion even follows without assuming finite variance, since you assumed the mean $\mu$ exists. The strong law of large numbers then give the result, it can be proved without assuming finite variance.

$\endgroup$
  • $\begingroup$ +1: thank you for improving the second conclusion. $\endgroup$ – whuber Jul 7 '18 at 22:48
2
$\begingroup$

One case in which $\hat \mu$ may be a biased estimator of $\mu$: the samples $x_1,..., x_n$ are not uniformly randomly sampled from the population of interest. This is really two problems:

(1) Some values in the population are more likely to be sampled than others. A classic example of this is when we are looking at a voluntary polls of opinion. It seems very probably that people with strong opinions are more likely to complete the survey than people who are more indifferent.

(2) We have samples, but they are not from the population of interest. This somewhat seems like a cop-out, but in practice this is extremely common. For example, when we look at polls before an election, the population of interest is the actual votes cast on election night. Clearly, it's not possible to get any of those samples before the election, so we look at a population that has a distribution we assume/hope is close to the real distribution we care about and can be sampled: polls taken ahead of the election...and then we also either hope that we have a uniform random sample from that population, or we use methods to attempt to rebalance that estimator due to over/under representation of various groups. We may also try to adjust for potential changes in opinion of the population over time to account for the fact that we don't have samples from the population of interest, but may be able to model some of the relation between the population of interest and population we can sample.

I suspect this may not be the type of answer you were looking for: i.e., I think the OP may have been curious about the case when $x_i$ were uniformly sampled from the correct distribution, but the estimator could have been inconsistent (as noted, this can happen if the variance is undefined). I presented this answer as I think it's a much more common issue in practice and often should be considered more closely during applied analyses.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.