We simulate $E(\exp(z))$ by Monte Carlo method,where $z\sim{}N(0,1)$. For sample sizes $2^{16}$ and $2^{17}$, the variance errors are 0.00531 and 0.00364, respectively. The ratio of these two errors is approximately $1.5$. My questions are:
(1) What information can we learn from this ratio?
(2) Why should the ratio approach $\sqrt 2$ as the sample size approaches infinity?
Having had a quick look; you may consider this first part of my answer a form of technical book review based on about 3 pages of the following book:
Desmond Higham (2004) An Introduction to Financial Option Valuation, Mathematics, Stochastics and Computation, Volume 13, Cambridge University Press.
It appears that this is the set up in the book:
$X_1, X_2 ...$ are (quote) "random variables" with mean $a$ and variance $b^2$; $a_M$ is the mean of the first $M$ $X$'s (okay, it makes sense to here, though the use of $a$ is odd because population parameters are more conventionally Greek letters). Now things get bizarre $b_M$ is, as defined, the sample variance. That is, $X_i$ has apparently but subtly changed from a random variable to an observation, presumably starting at least back at the definition of $a_M$.
So that's got no chance of being anything but confusing. Let's start again with less misleading notation and terminology. Let me set up some terms, so at least I have some idea what I am talking about:
Let $X_1, X_2 ...$ be independent, identically distributed random variables with common mean $\theta$ and variance $\tau^2$.
Let $x_1, x_2 ...$ denote observations on those random variables. Let $\bar{x}_m$ be the sample mean of the first $m$ $x$'s and let $s^2_m$ be the sample variance of the first $m$ $x$'s.
(The author confused those two things together. Not okay.)
The author then invokes the Central Limit Theorem to say that $\bar{x}_m - \theta$ should be approximately $\sim N(0,\tau^2/m)$. Not technically the right way to do it, but I will say okay, up to some caveats and handwaving.
He now calls $\tau/\sqrt{m}$ 'the standard error'. It is in fact the standard error of the sample mean. Anyway, let's move on.
He now refers to $s^2_m$ as "the variance approximation". In fact, right now it's just the sample variance, but of course, the sample variance (under certain conditions) will in large samples be an approximation for $\tau^2$. He has the cart before the horse.
He now implies he can use $s_m$ for $\tau$ in his earlier discussion about the distribution of the deviation in the mean. Well, yes, but you have to do it right. By invoking Slutsky's theorem, we can have that $(\bar{x}_m - \theta)/(s_m/\sqrt{m})$ will go to a standard normal as $m \rightarrow \infty$. Which isn't quite what he said, but let's allow him some significantly more vigorous handwaving.
Now, finally, we get the definition of "the variance error". Translating, it's $|s^2_m - \tau^2|$. Okay, so what he actually means is the absolute deviation of the sample variance from the population value. Let's denote that by $d_m$.
The conclusion of the review and translation: The author is trying to explain something important, but did a very poor job of it
--
So now the question is, if $m_1$ is some large sample size, and $m_2$ is say $2m_1$, if we compute $d_{m_1}/d_{m_2}$, why does that ratio approach $\sqrt 2$ and what does that tell us?
As the author explained a few pages before this point, the variance of an average of iid random variables decreases with the sample size. This is true also of the sample variance, itself a kind of average (an average of squared differences - I must excuse my own little bit of handwaving of details to do with the difference between $m-1$ and $m$ on the denominator; this can be made technically correct and still go through). As such, its own variance decreases with increasing $m$. The quantity $(s_m^2 - \tau^2)/(1/\sqrt{m})$ is (as $m$ becomes large) itself going to have approximately a normal distribution with mean zero and constant variance.
So, if $r_1= |s_{m_1}^2 - \tau^2|/(1/\sqrt{m_1})$ and $r_2$ is similarly defined, then $r_1/r_2$ will (again, by invoking Slutsky!) converge to 1.
Now $d_1/d_2 = \sqrt{m_2/m_1}\cdot r_1/r_2$. So that converges to $\sqrt 2$.
As for what it tells you, merely that the payback for twice the work isn't twice as much accuracy; the sample variance approaches the population variance in the same fashion as the sample mean does - fairly slowly, as $1/\sqrt{m}$.
Well an important question is how many samples do you need to half the sampling error. essentially by the central limit theorem, the sample mean, has standard error sigma/sqrt(N), [where sigma is the "true" population standard deviation]
so all the ratio is telling you is that doubling the number of samples reduces the error by only 1/sqrt(2)...
