Number of samples needed in Monte Carlo simulation: how good is this approximation? In Risk Theory Beard, Pentikanen and Pesonen (1969) mention a method of assessing number of samples needed for Monte Carlo simulation as
$$ \sigma = \sqrt{\frac{p(1-p)}{s}} \leq \frac{1}{2} \sqrt{ \frac{1}{s}} $$
where $F(x) = p$, i.e. it is a probability of observing some value $x$ and $s$ is a number of samples. This shows us that with 99% confidence value we can expect that values observed in simulation study will lie $\pm 2.576 \sigma$ from $p$'s. This is similar to simulation standard error estimation based on observed variance mentioned by Aksakal. The authors seem to suggest that the formula can be used before the simulation to assess number of samples needed ($s$) to obtain simulated results with some precision of interest.
How good is this approximation?
 A: The approximation could be poor when $p$ is close to zero or one, but when $p = 1/2$ it holds exactly.
The idea here is that we want to estimate the probability of an event by using a sample proportion across many Monte Carlo trials, and we want to know how accurate of an estimate that proportion is of the true probability.  The standard deviation $\sigma$ of a sample proportion is as the authors note $\sqrt{p (1 - p) / s}$ (where $s$ is the number of Monte Carlo simulations), but the problem is we don't know $p$.  However, we can maximize $\sigma$ with respect to $p$ and get a conservative "estimate" of this standard error that will always hold no matter what $p$ happens to be.  This may end up causing us to run more simulations than we need to, but this won't matter as long as the iterations themselves are computationally cheap.
A: This approximation is called Wald confidence interval. It's based on normal approximation of Binomial. How good is this approximation? There are two answers: when sample size is at least 30 and "it depends".
The "30" answer is very popular and has been propagated from book to book until it became an axiom, pretty much. Once I was able to track the first paper which mentioned it.
The "it depends" answer is explored in this paper: Central Limit Theorem and Sample Size by Zachary R. Smith and Craig S. Wells 
The other thing is variance reduction. It is especially critical to use for low or high $p$. Obviously, Wald's formula will not work directly in this case.
