9
$\begingroup$

I am trying to estimate the mean of a more-or-less Gaussian distribution via sampling. I have no prior knowledge about its mean or its variance. Each sample is expensive to obtain. How do I dynamically decide how many samples I need to get a certain level of confidence/accuracy? Alternatively, how do I know when I can stop taking samples?

All the answers to questions like this that I can find seem to presume some knowledge of the variance, but I need to discover that along the way as well. Other are geared towards taking polls, and it's not clear to me (beginner that I am) how that generalizes -- my mean isn't w/in [0,1], etc.

I think this is probably a simple question with a well known answer, but my Google-fu is failing me. Even just telling me what to search for would be helpful.

$\endgroup$
8
  • $\begingroup$ Any reason why you marked this as CW? The question seems specific enough to allow for one right answer and thus should not be CW. $\endgroup$ – user28 Sep 20 '10 at 14:24
  • 1
    $\begingroup$ @josh that's fine. I was just curious about your choice. $\endgroup$ – user28 Sep 20 '10 at 18:25
  • 1
    $\begingroup$ Google "adaptive sampling" and "sequential sampling". If you're still stuck, include "Wald" as a keyword and then work forward historically (i.e, look at papers that reference Wald's work on sequential sampling, then look at papers that reference them, etc.). $\endgroup$ – whuber Sep 20 '10 at 19:10
  • 1
    $\begingroup$ @Robby McKilliam: But what data do you use? This question arises before any data have been collected. If you collect values one at a time and compute a CI after each new one is added to the dataset, you cannot use standard formulas for the intervals due to the correlated multiple comparisons you are making. Thus, you need a stopping rule that optimizes the sum of the statistical risk of your estimator and the cost of collecting each additional sample. $\endgroup$ – whuber Sep 20 '10 at 21:22
  • 1
    $\begingroup$ @whuber thanks! I'm still digesting the material, but I think that this is exactly what I'm looking for. If this were an answer, I'd accept it... $\endgroup$ – Josh Bleecher Snyder Sep 20 '10 at 21:41
2
$\begingroup$

You need to search for 'Bayesian adaptive designs'. The basic idea is as follows:

  1. You initialize the prior for the parameters of interest.

    Before any data collection your priors would be diffuse. As additional data comes in you re-set the prior to be the posterior that corresponds to the 'prior + data till that point in time'.

  2. Collect data.

  3. Compute the posterior based on data + priors. The posterior is then used as the prior in step 1 if you actually collect additional data.

  4. Assess whether your stopping criteria are met

    Stopping criteria could include something like the 95% credible interval should not be bigger than $\pm \epsilon$ units for the parameters of interest. You could also have more formal loss functions associated with the parameters of interest and compute expected loss with respect to the posterior distribution for the parameter of interest.

You then repeat steps 1, 2 and 3 till your stopping criteria from step 4 are met.

$\endgroup$
0
$\begingroup$

You would normally want at least 30 to invoke central limit theorem (though this is somewhat arbitrary). Unlike in the case with polls etc, which are modelled using the binomial distribution, you can not determine a sample size beforehand which guarantees a level of accuracy with a Gaussian process - it depends on what residuals you get which determine the standard error.

It should be noted that if you have a robust sampling strategy, you can get much more accurate results than with a much larger sample size with a poor strategy.

$\endgroup$
2
  • 3
    $\begingroup$ Why would one need to invoke the CLT when sampling from a known (or assumed) Gaussian distribution? The mean of even a sample of one will be Normally distributed! $\endgroup$ – whuber Sep 20 '10 at 19:09
  • $\begingroup$ Good point! Didn't RTQ properly. $\endgroup$ – James Sep 21 '10 at 9:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.