# Combine multiple Monte-Carlo estimates

I use a Monte Carlo simulation (say 100.000 runs) to estimate parameter in R. I have memory problems and my first thought is to run multiples times my estimation program (say 500 times) .

My question are: what is the distribution of an estimator ? how to get an estimator from multiple estimators ? (for the mean you can use a mean of means for example, does this hold in general ?). Is it possible to derive some kind of confidence interval from the runs ?

My questions are both general and specific to quantile and correlation estimation. For the quantile and correlation part thanks to assume a general distribution.

• Your description is pretty vague but it seems that is is just a programming issue with efficient using of memory (i.e. not storing all the simulation output in RAM but taking only the important or aggregated values, saving things on disc rather that storing in memory, in most cases you can even stop a simulation and start again in the place you stopped)
– Tim
Nov 13, 2015 at 15:49
• Once I understand how to aggregate estimator for quantile and correlations I may be able to use inline calculation. Like after n simulation, a run r, and one estimator of the mean m I can use (n*m + r)/(n+1). For the moment the problem is I don't know how to aggregate quantile and correlation estimators (well in fact thinking about correlation, it look like a mean). Nov 13, 2015 at 15:57

The theoretical basis of Monte Carlo estimation is the Law of Large Numbers. This means that the empirical cdf $$\hat{F}_i(x)=\dfrac{1}{n}\sum_{i=1}^n \mathbb{I}_{x_i\le x}$$ is an unbiased estimator of the true cdf $F$. Unfortunately, you cannot store this empirical cdf $\hat{F}_i$ without storing the entire simulation sample of the $n$ $x_i$'s. However, if you replace $\hat{F}_i$ with an approximation based on the empirical percentiles $$\hat{\hat{F}}_i(x)=\dfrac{1}{99}\sum_{i=1}^{99} \mathbb{I}_{\hat{c}_i\le x}$$where $\hat{c}_i$ is the $i$-th empirical percentile, given by $$\hat{F}_i(\hat{c}_i)=\dfrac{i}{100}$$ you only need to store $99$ values. (If the percentile precision is not sufficient, you can move to the permile precision or beyond.) You can then repeat simulations without undue pressure on storage by averaging the $\hat{\hat{F}}_i$'s. From this average, you can derive a convergent estimator of the quantiles of interest.
As you noticed, since $\text{cor}(X,Y)$ can be consistently estimated as $$\dfrac{\frac{1}{n}\sum_{i=1}^n x_iy_i -\frac{1}{n}\sum_{i=1}^n x_i\,\frac{1}{n}\sum_{i=1}^n y_i}{\left\{\frac{1}{n}\sum_{i=1}^n x_i^2-\frac{1}{n^2}(\sum_{i=1}^n x_i)^2\right\}^{1/2}\left\{\frac{1}{n}\sum_{i=1}^n y_i^2-\frac{1}{n^2}(\sum_{i=1}^n y_i)^2\right\}^{1/2}}$$you can easily update the five quantities involved in this expression $$\sum_{i=1}^n x_iy_i,\ \sum_{i=1}^n x_i,\ \sum_{i=1}^n y_i,\ \sum_{i=1}^n x_i^2,\ \sum_{i=1}^n y_i^2$$ without increase storage pressure.