0
$\begingroup$

Suppose I want to use Monte Carlo to compute some probability $p$. A single MC simulation will run for $R$ iterations and calculate $p$ as the fraction of 'successes'.

Say I want to compute $p$ within an error of $E$ with a 95% confidence interval. That is, I want to find $R_0$ such that if I run the MC simulation for $R_0$ many times and obtain $p_0$, then I am $95\%$ confident that the true $p$ lies in $[p_0 - E, p_0 + E]$.

I found two possible formulas for this: one and two but they are different (albeit similar), and they also don't really seem to take $R$ into account, which doesn't make intuitive sense to me.

For instance, the second link has the formula:

$$\bigg(\frac{z_{\alpha/2} \cdot \text{std}(p)}{E}\bigg)^2$$

$\text{std}(p)$, I assume will be computed by Monte Carlo sampling $p_1, \dots, p_n$ (with some fixed $R$ iterations for each $p_i$), and the finding the standard deviation of the $p_i$. But naturally this standard deviation would decrease as $R$ increases. So it seems to me that the formula should factor in $R$ somehow, which it isn't.

Is my interpretation incorrect?

Is there a simple formula to determine number of simulations required?

Improve this question
$\endgroup$
9
  • $\begingroup$ Welcome to Cross Validated! Is this anything more than finding the sample size required to detect an effect of $E?$ // If you use R software, the pwr package has some nice functions for doing work like this. // Are you sure you want to discuss the margin of error in additive terms? While people do this all the time, if you observe a probability of $0.1$, a margin of error of $0.05$ is a lot bigger deal than if you observe a probability of $0.5$. $\endgroup$
    – Dave
    Commented Nov 12, 2022 at 20:01
  • $\begingroup$ I'm using python - anything in scipy or numpy or statsmodels?. I'm not sure what "effect of $E$ means though. And yeah, it is additive but $E = 0.001$ and $p~ 0.55$ so it should be fine. It's a hypothetical/conceptual question, I don't intend to actually run the simulation for that long (I assume it would take quite long for such small errors) $\endgroup$
    – Hullo
    Commented Nov 12, 2022 at 20:13
  • $\begingroup$ $R$ is the number of iterations, which is the quantity you aim to calculate. $\endgroup$
    – Dave
    Commented Nov 12, 2022 at 23:30
  • $\begingroup$ How would I calculate $std(p)$ then? I have no knowledge of the underlying dist. $\endgroup$
    – Hullo
    Commented Nov 12, 2022 at 23:32
  • $\begingroup$ You know it’s a Bernoulli distribution. The variance of a Bernoulli distribution is $p(1-p)$. $\endgroup$
    – Dave
    Commented Nov 12, 2022 at 23:41

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.