I have a set $S$ of $N$ real numbers. I wanna calculate the mean $m$ variance $v$ of $S$. But since $N$ is too large, I do not want to use all of the numbers. Instead, I would like to sample $n$ numbers uniformly randomly from the $N$-number set $S$, such that mean and variance of the sampled set does not deviate too much from the original set. Say, the error I can tolerate is $e_m$, and $e_v$ respectively (which means that the mean of the sample set should fall in $(1 \pm e_m)m$, and the sample variance should fall in $(1\pm e_v)v$). How large should $n$ be?
Tell me more
×
Cross Validated is a question and answer site for
statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
|
|
|||||||||
|
|
Statistics doesn't allow for absolute certainty in estimating moments (like the mean and variance). What you can do is set a confidence level (say, 95%) and determine the sample size n necessary to be 95% certain that the mean is within your tolerance. You'll need to know something about the variance of the distribution to use the equations here: Wikipedia's Sample Size. Depending on how rigorous you need to be about this, you can probably start by using a similar approach to estimate the variance, and then use the estimated variance in calculating n for the mean. Obviously, you could do this iteratively, getting better results with each iteration. Generally, straight-forward and and informationally efficient estimates require you to know (or assume) some things about your set S, like how it is distributed. |
|||
|
|
