How do I analyze a simulation study? Say I want to simulate from a density $f$. 
I come up with two procedures for doing so, $proc1$ and $proc2$. 
Now, I can obviously plot the histograms that come from simulating from $proc1$ and $proc2$ and plot $f$ on top of them, and say "yeah looks like they both work".
But what more can I say? Preferably, I want to be able to say something about the accuracy of those procedures. I want to be able to compare them and say something about which one is better or worse.
What can typically be done in this situation?
Right now, my only idea is to calculate the sample mean from both $proc1$ and $proc2$, and calculate its standard error, and then the simulation with a smaller error is better?
 A: Depending on your needs, some characteristics per sample ($\times 10^n$ as appropriate) of simulated $f$ (plus the CDF, $F=\int f$, and quantile function $F^{-1}$) that may be of interest for procedure 1 vs procedure 2:


*

*Computational time (e.g., measured using clock cycles, seconds, or represented analytically using big O notation)

*Memory cost

*Energy cost

*Uniformity of $p$ (from $F^{-1}$)

*Accuracy in the tails of $f$

*Accuracy for the distribution overall from an analytic formulation of $f$ (e.g., using a one-sample Kolmogorov-Smirnof test for each procedure for many samples of size $N$ and comparing the mean $p$ values).
The particular statistics tracked from simulating a univariate distribution will necessarily depend on the purposes envisioned for using it. For example, if the number of needed simulations is anticipated to be on the order of, say, $10^6$ per day or less, then you may not need to calculate compute time, memory, or energy costs. However, if anticipated need for simulated draws from $f$ is at a much greater rate, say $10^9$ per hour or more, then those costs per sample may be relevant.
