1
$\begingroup$

I am running an experiment and I would like what is a good method to calculate how many times I need to run an experiment, in other words, how many data points I will need to collect. The issue is that I do not know my population. Theoretically speaking, I could repeat the measurement many times (assuming no budget or time/budget or other constraints). On the other hand, I know the confidence interval that I'd like to have (95%) and a margin of error of no more than 5%

Assume no prior knowledge about the mean or standard deviation (essentially I will be running this experiment for the 1st time).

Any ideas or suggestions?

$\endgroup$
4
  • $\begingroup$ Confidence interval for what? $\endgroup$
    – Dave
    Dec 13, 2019 at 3:03
  • $\begingroup$ For the statistic of interest (e.g. mean) for a metric. $\endgroup$
    – gplt
    Dec 13, 2019 at 7:27
  • $\begingroup$ It may be worth your time to look into power. The statistical power of a hypothesis test is, among other things, related to the number data points (number of subjects in the analysis, for instance). $\endgroup$ Dec 14, 2019 at 5:49
  • $\begingroup$ Hi Adam, I thought about this but I am not dealing with hypotheses in my case. I am dealing with a continuous variable. $\endgroup$
    – gplt
    Dec 16, 2019 at 19:37

1 Answer 1

1
$\begingroup$

It appears you want to know how many samples you need to estimate a mean to a given precision with a given confidence but that you don't have a hypothesis to test.

To answer this, you need to do a power analysis. This requires an estimate of the standard error, which you can get from N and the standard deviation. But you need to estimate the SD. You can base this estimate on prior knowledge, a literature review or whatever, but you need to estimate it somehow.

$\endgroup$
2
  • $\begingroup$ Well, it depends on how you define power analysis. The ideas are similar. You need to estimate sd. $\endgroup$
    – Peter Flom
    Dec 16, 2019 at 20:04
  • 1
    $\begingroup$ Hi Peter, yes, that is correct. I looked into power analysis but this approach requires having some sort of hypothesis which I don't have (and want to avoid if possible). What I think is right for my case is the following formula for the margin of error: $$ \text{margin of error} = z \frac{\sigma}{\sqrt{n}} $$ where z is the z-score or standard score and which I solve for n and assuming a SD from prior knowledge. Happy to know if there are any alternative or better ways! $\endgroup$
    – gplt
    Dec 16, 2019 at 20:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.