Error bars on log of big numbers

I am calculating a quantity of the following form:

$\mu = \log( \frac{1}{n} \sum_{i=1}^{n} e^{\phi(X_i)} )$

via MC. $X_i$ are iid and I can sample them. I want to give error bars\ confidence interval on $\mu$.

One approach is to take a normal approximation to the sum, get error bars on the sum and transform using the logarithm, like so:

$\mu \in ( \log(\mu - 2\sigma) , \log( \mu + 2\sigma) )$,

where $\sigma$ is the std of $e^{\mu}$. Is this a reasonable thing to do? Is there a better way?

Using the above there's another problem. $\phi(X_i)$ might become so big to the point that $e^{2\phi(X_i)}$ overflows. This renders summing and squaring for calculating the error bars impossible. The method with which I calculate the variance doesn't seem to matter:

$\sigma^2 = \frac{1}{n}[ \bar{ x^2 } - \bar{x}^2]$

is just as bad as

$\sigma^2 = \frac{1}{n}[\frac{1}{n}\sum (x_i - \bar{x})^2]$ ($x_i$ are not $X_i$)

since in this scenario the sample mean is small compared to the largest value attained (this might imply the normal approximation is unjustified, but I would still like to have some measure of goodness for my estimate).

One possible solution I considered is to take a "large deviations" approach and just consider the most significant (biggest) sample:

$\sigma = \sqrt{\frac{1}{n}[\frac{1}{n}\sum (x_i - \bar{x})^2]} = \frac{1}{n} \sqrt{\sum (x_i - \bar{x})^2} \approx \frac{1}{n} (\max_i(x_i) - \bar{x})$

which in my case would give:

$\sigma \approx \frac{1}{n} [\max_i(e^{\phi(X_i)}) - e^{\mu}]$

I would like to know if anyone has a better solution.

The problem isn't as profound as it may appear. Because

$$\frac{1}{n} \sum_{i=1}^{n} e^{\phi(X_i)} = e^Y\frac{1}{n} \sum_{i=1}^{n} e^{\phi(X_i)-Y}$$

for

$$Y = \max_i\{\phi(X_i)\},$$

an algebraically equivalent expression is

$$\mu = Y -\log n + \log \sum_{i=1}^{n} e^{\phi(X_i)-Y}.$$

In this one there will be no difficulties computing the log of the sum, which necessarily lies between $1$ and $n$ (since all the exponents are non-positive). In particular, there is no chance of overflow and any underflow will be absorbed (to high precision) in the summation, where at most $\log_2 n$ bits will be lost (and almost certainly the loss in precision will be less than around $\frac{1}{2}\log_2 n$ bits). If you have any concern about precision losses, sum the terms in ascending order of $\phi(X_i)$.

The same approach applies to computing the moments needed to obtain an estimated standard deviation.

Using a normal approximation may be unwise unless you are sure that all the $\exp(\phi(X_i))$ will be well within an order of magnitude of each other (which means the $\phi(X_i)$ should all lie within an interval of around $2$ or less). Even then you might need a fairly large value of $n$. If just a few of those values dominate the others, then the averaging-out that justifies this approximation will not occur, regardless of the size of $n$.

• Will this work if $\phi(t)=e^t$? – Aksakal Dec 11 '14 at 18:15
• @Aksakal Sure, why not? The only implicit assumption in this analysis--as in all computing-related questions on this site!--is that $\phi$ will not overflow or underflow when applied to the values of $X_i$ that are generated. – whuber Dec 11 '14 at 18:22
• The meaning of error bars is usually related to a standard deviation. So I thought if $\mu$ doesn't have one, then maybe there could be an issue with error bars too. – Aksakal Dec 11 '14 at 18:24
• @Aksakal The data will always have a finite standard deviation. My answer is not attempting to address the process of estimating error bars (which can automatically be performed by the MC algorithm), but only to show how to get the algorithm to work by avoiding overflow. – whuber Dec 11 '14 at 18:29
• you are right, you can compute std of even Cauchy distribution with MC. I just thought OP must be aware that his distribution should be well behaving for this to make a sense. – Aksakal Dec 11 '14 at 18:36