I am currently using a Latin Hypercube Sampling (LHS) to generate well-spaced uniform random numbers for Monte Carlo procedures. Although the variance reduction that I obtain from LHS is excellent for 1 dimension, it does not seem to be effective in 2 or more dimensions. Seeing how LHS is a well-known variance reduction technique, I am wondering whether I may be misinterpreting the algorithm or misusing it in some way.
In particular, the LHS algorithm that I use to generate $N$ spaced out uniform random variables in $D$ dimensions is:
For each dimension $D$, generate a set of $N$ uniformly distributed random numbers $\{u^1_D,u^2_D...u^N_D\}$ such that $u^1_D \in [0,\frac{1}{N+1}]$, $u^2_D \in [\frac{1}{N+1}, \frac{2}{N+1}]$ ... $u^N_D \in [\frac{N}{N+1}, 1]$
For each dimension $D \geq 2$, randomly reorder the elements from each set. The first $U(0,1)^D$ produced by LHS is the a $D$ dimensional vector containing the first element from each reordered set, the second $U(0,1)^D$ produced by LHS is the a $D$ dimensional vector containing the second element from each reordered set, and so on...
I have included some plots below to illustrate the variance reduction I get in $D = 1 $ and $D = 2$ for a Monte Carlo procedure. In this case, the problem involves estimating the expected value of a cost function $E[c(x)]$ where $c(x) = \phi(x)$, and $x$ is a $D$-dimensional random variable distributed between $[-5,5]$. In particular, the plots show the mean and the standard deviation of 100 sample mean estimates of $E[c(x)]$ for sample sizes of 1000 to 10000.
I get the same type of variance reduction results regardless of whether I use my own implementation or the lhsdesign function in MATLAB. Also, the variance reduction does not not change if I permute all sets of random numbers instead of just the ones corresponding to $D \geq 2$.
The results make sense since stratified sampling in $D = 2$ means that we should sample from $N^2$ squares instead of $N$ squares that are guaranteed to be well spread.
