Combine multiple Monte-Carlo estimates I use a Monte Carlo simulation (say 100.000 runs) to estimate parameter in R. I have memory problems and my first thought is to run multiples times my estimation program (say 500 times) . 
My question are: what is the distribution of an estimator ? how to get an estimator from multiple estimators ? (for the mean you can use a mean of means for example, does this hold in general ?). Is it possible to derive some kind of confidence interval from the runs ?
My questions are both general and specific to quantile and correlation estimation. For the quantile and correlation part thanks to assume a general distribution.
 A: Only the mean of means allows for a strict recovery of multiple simulations with limited storage capacities. However, Monte Carlo being an approximation method, approximations are always available.
The theoretical basis of Monte Carlo estimation is the Law of Large Numbers. This means that the empirical cdf
$$\hat{F}_i(x)=\dfrac{1}{n}\sum_{i=1}^n \mathbb{I}_{x_i\le x}$$
is an unbiased estimator of the true cdf $F$. Unfortunately, you cannot store this empirical cdf $\hat{F}_i$ without storing the entire simulation sample of the $n$ $x_i$'s. However, if you replace $\hat{F}_i$ with an approximation based on the empirical percentiles
$$\hat{\hat{F}}_i(x)=\dfrac{1}{99}\sum_{i=1}^{99} \mathbb{I}_{\hat{c}_i\le x}$$where $\hat{c}_i$ is the $i$-th empirical percentile, given by
$$\hat{F}_i(\hat{c}_i)=\dfrac{i}{100}$$
you only need to store $99$ values. (If the percentile precision is not sufficient, you can move to the permile precision or beyond.) You can then repeat simulations without undue pressure on storage by averaging the $\hat{\hat{F}}_i$'s. From this average, you can derive a convergent estimator of the quantiles of interest.
As you noticed, since $\text{cor}(X,Y)$ can be consistently estimated as $$\dfrac{\frac{1}{n}\sum_{i=1}^n x_iy_i -\frac{1}{n}\sum_{i=1}^n x_i\,\frac{1}{n}\sum_{i=1}^n y_i}{\left\{\frac{1}{n}\sum_{i=1}^n x_i^2-\frac{1}{n^2}(\sum_{i=1}^n x_i)^2\right\}^{1/2}\left\{\frac{1}{n}\sum_{i=1}^n y_i^2-\frac{1}{n^2}(\sum_{i=1}^n y_i)^2\right\}^{1/2}}$$you can easily update the five quantities involved in this expression 
$$\sum_{i=1}^n x_iy_i,\ \sum_{i=1}^n x_i,\ \sum_{i=1}^n y_i,\ \sum_{i=1}^n x_i^2,\ \sum_{i=1}^n y_i^2$$
without increase storage pressure.
