# Error Bars for Monte Carlo Experiment

Suppose we have a random variable $X$, where $\mathbb{E}(X)$ and $\text{Var}(X)$ are known. I have computed $N$ number of MC-type samples from the distribution of $X$.

Let $\bar{x} = \frac{1}{N}\sum x_i$ where $x_i$ represents a single sample from our distribution. From CLT, I know that the error bars for $\bar{x}$ can be constructed in the following way, assuming 90% CI: $$\bar{x} \pm (1.64)\sqrt{\frac{\text{Var}(X)}{N}}$$

But I'm interested in computing the error bars for the relative error $\frac{|\bar{x} - \mathbb{E}(X)|}{|\mathbb{E}(X)|}$. Since the relative error itself is a random variable, I'm having difficulty constructing error bars for it .

I wonder if I can use relative error in Variance and substitute that with $\text{Var}(X)$.

• $\mu$ is by convention used to denote the population mean ($\mu=E(X)$) while $\bar X$ would denote the random variable respresenting a sample mean (and $\bar x$ would denote an observed sample mean). Indeed, by convention, Greek letters are mostly kept for population parameters and Roman letters are mostly kept for sample quantities (with a few somewhat common exceptions). It's likely that people will find your unconventional notation confusing in this particular case, so you should consider whether it might be better to come closer to the usual convention. – Glen_b Jun 13 '14 at 22:34
• I assume we can take $E(X)\neq 0$ as given. If it does tend to be even in the region of 0 (i.e. that it might sometimes be one side and sometimes another, and in either case might come close in some sense), I'm not convinced of the wisdom of looking at the relative error. By contrast, if $X$ is a positive r.v., relative error often makes a lot of sense. Is $E(X)$ somewhat likely to be negative? – Glen_b Jun 13 '14 at 22:38
• Correct $E(X)$ is not equal to zero and it is never negative. – David Jun 13 '14 at 22:39

Writing $\mu=E(X)$ and $\sigma^2=\text{Var}(X)$

Let's assume (as you did) that the distribution of $X$ is one for which the CLT approximately "kicks in" by sample size $n$. That is $\bar{X} \,\,:\hspace{-.58em}\sim N(\mu,\sigma^2/n)$.

Note that $Y=\bar{X}-\mu$ will have mean 0 and be approximately normal, so $|Y|$ has a scaled chi distribution.

If $Z = \sqrt{n} Y/\sigma$, then $|Z|$ has a standard chi-distribution:

Since the distribution is skew, we won't have any kind of symmetric interval, so it doesn't really make sense to write it in $\pm$ form.

The 5th percentile of this distribution is the 52.5 percentile of the corresponding standard normal, which is at 0.0627, which the 95th percentile is at the 97.5 percentile of the standard normal, 1.96. That is, 90% of the standard chi lies in $(0.0627,1.96)$.

So we have:

$0.0627<|Z|<1.96$

$0.0627<\frac{|Y-\mu|}{\sigma/\sqrt{n}}<1.96$

$0.0627\frac{\sigma}{\mu\sqrt{n}}<\frac{|Y-\mu|}{\mu}<1.96\frac{\sigma}{\mu\sqrt{n}}$

with approximate 90% probability.

Hence $(0.0627\frac{\sigma}{\mu\sqrt{n}},1.96\frac{\sigma}{\mu\sqrt{n}})$ will form an approximate 90% interval for $\frac{|\bar{X}-\mu|}{\mu}$.

The mean of $\frac{|\bar{X}-\mu|}{\mu}$ will be $\sqrt{\frac{2}{\pi}}\frac{\sigma}{\mu\sqrt{n}}\approx 0.7979\frac{\sigma}{\mu\sqrt{n}}$ and the median will be $\approx 0.6745\frac{\sigma}{\mu\sqrt{n}}$

($\sigma/\mu$ is sometimes called the coefficient of variation)