2
$\begingroup$

I have a bunch of IID samples from a random variable with unknown mean and unknown variance. I now need to know the variance of the average of these samples.

I found some references. In decreasing order of helpfulness: 1, 2, 3. These give a formula for the variance in terms of the variance of the original distribution. What I haven't been able to find is a direct solution for the case of the original distribution having unknown variance. This similar question may be of use, but I am unsure how to apply it.

My first thought is to try to use the sample variance as an estimate for the population variance. My question is, is that the right way to do it? Is so, how do I correct for any differences? If not, what is the right way?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

The question you think it might be "similar", is not, because it is concerned mainly with the distribution/variance of the sample variance itself.

You cannot know the variance of the sample mean (i.e. of the "average of these samples" - I guess by "sample" you mean a single observation) because it depends on a usually unknown parameter (the population variance). So indeed, what you can do is use in its place the estimated variance, i.e. the sample variance, corrected for bias:

$$s^2 = \frac1{n-1} \sum(x_i-\bar x)^2, \;\; E(s^2) = \sigma^2$$

As for the question

How do I correct for any differences?

The answer is "you cannot" -and this is the whole point of statistics. If you could "correct for any differences", it would imply that you already knew the true variance, isn't it so?

Improve this answer
$\endgroup$
5
  • $\begingroup$ "How do I correct for any differences?": to clarify, I meant, "I need to correct for bias in the sample variance the same way I normally do, right?" Just to be crystal clear, I can just plug in $s^2$ as above into the formula $\sigma^2/n$ to get $E[Var(\bar{x})]=s^2/n$, right? $\endgroup$
    – imallett
    Nov 3, 2014 at 0:51
  • 1
    $\begingroup$ $s^2/n$ is the estimated variance of the sample mean, not the variance's expected value. What you do have is that $E[\hat Var(\bar x)]= E(s^2/n) = \sigma^2/n$, since you use the bias-corrected sample variance expression. $\endgroup$
    – Alecos Papadopoulos
    Nov 3, 2014 at 0:56
  • $\begingroup$ Oh right; I meant that $s^2 \rightarrow \sigma^2$ as $n$ increases. Your correction is equivalent, I believe. Thanks, $\endgroup$
    – imallett
    Nov 3, 2014 at 1:01
  • $\begingroup$ Yes it is also a consistent estimator. $\endgroup$
    – Alecos Papadopoulos
    Nov 3, 2014 at 1:12
  • $\begingroup$ I think the second comment above really answers the original question: $s^2/n$ is a point estimator for the variance of the sample mean, and it's unbiased. $\endgroup$
    – flow2k
    Jan 10, 2021 at 12:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.