0
$\begingroup$

I have a simulation that returns "yes" or "no" for each iteration, and I measure the average number of "hits" over many iterations to estimate the likelihood of "yes" occurring. I'd like to be able to say how accurate/trustworthy that estimate of the likelihood is, though.

Monte Carlo integration explains a similar basic Monte Carlo simulation:

Imagine that we want to measure the area of a pond with arbitrary shape. Suppose that this pond is in the middle of a field with known area $A$. If we throw $N$ stones randomly, such that they land within the boundaries of the field, and we count the number of stones that fall in the pond $N_{in}$, the area of the pond will be approximately proportional to the fraction of stones that make a splash, multiplied by $A$:

$$A_{pond}=\frac{N_{in}}{N}A.$$

That page then talks about a similar scenario, where you do integration of a function by checking whether random points are under it or not:

imagine a rectangle of height $H$ in the integration interval $[a,b]$, such that the function $f(x)$ is within its boundaries. … The fraction of points that fall within the area contained below $f(x)$ … is an estimate of the ratio of the integral of $f(x)$ and the area of the rectangle.

But then it switches to a different method of integration, where instead of checking yes/no for whether your test points falls under the curve, you instead evaluate the curve itself:

Another Monte Carlo procedure is based on the definition: $$\langle g \rangle=\frac{1}{(b-a)} \int_a^b{f(x)dx}.$$

In order to determine this average, we sample the value of $f(x)$:

$$\langle f \rangle \simeq \frac{1}{N}\sum_{i=1}^{N}f(x_i),$$

And then it tells how to calculate the variance for that type of method:

A possible measure of the error is the "variance" $\sigma^2$ defined by: $$\sigma ^2=\langle f^2 \rangle - \langle f \rangle ^2, $$

But I don't understand how to take this variance calculation and transfer it back to the original "number of hits" type of calculation. I'm not sure what function $f$ I'm evaluating or how to evaluate it.

Improve this question
$\endgroup$
11
  • 1
    $\begingroup$ I am unable to match your description to the page you link to. Regardless, this is a Bernoulli experiment and the errors are exactly described by Binomial distributions. You can read a huge amount about this situation on our site. With this search I found some details at stats.stackexchange.com/questions/151163: would this answer your question? $\endgroup$
    – whuber ♦
    Commented May 24, 2022 at 20:17
  • $\begingroup$ @whuber No, I don't understand that question or how it applies to mine. :/ $\endgroup$
    – endolith
    Commented May 24, 2022 at 21:42
  • $\begingroup$ @whuber This result from your search seems applicable though stats.stackexchange.com/a/71228/11633 $\endgroup$
    – endolith
    Commented May 24, 2022 at 21:48
  • 1
    $\begingroup$ That's incorrect: $A$ is an area while the $N_{in}$ and $N$ are counts: you can hardly subtract a count from an area! The estimator of the area is $\hat\theta = AN_{in}/N,$ which does make sense as a fraction of an area. $\endgroup$
    – whuber ♦
    Commented May 25, 2022 at 21:08
  • 1
    $\begingroup$ That looks correct. In fact, a suggestive way to write your formula is the form $$A^2\left(\frac{N_{in}}{N}\right)\left(1 - \frac{N_{in}}{N}\right)/N$$ because the factors have clear interpretations: $A^2$ arises by using the base area $A$ as the unit of measurement; $1/N$ arises in the usual way from an average of $N$ independent observations; and the rest has the familiar form $(\hat p)(1-\hat p)$ for the variance of a Bernoulli$(\hat p)$ variable. In short, this is the usual formula for the (squared) standard error of an estimate of $\theta$ for Binomial sampling. $\endgroup$
    – whuber ♦
    Commented May 26, 2022 at 13:32

0

Reset to default

Browse other questions tagged or ask your own question.