General Question

If someone handed you a direct sampling algorithm and a density function, and they told you that the two corresponded to the same random variate, how would you check this?


Some use cases:

  • Let $\mathbf{X}\in E\subseteq\mathbb{R}^n$ be a random vector whose density is $f_{\mathbf{X}}$ and for which a direct sampler is known. Let $\mathbf{g}:E\to\mathbb{R}^n$ be a "nice" transformation ($C^1$ diffeomorphism), and define a new random vector $\mathbf{Y}=\mathbf{g}(\mathbf{X})$. Since we can sample $\mathbf{X}$, it's easy to sample $\mathbf{Y}$; draw an $\mathbf{X}$ and apply $\mathbf{g}$ to it. Furthermore, you can derive the density of $\mathbf{Y}$ by computing $$f_{\mathbf{Y}}(\mathbf{y}) = f_{\mathbf{X}}(\,\mathbf{g}^{-1}(\mathbf{y})\,)\cdot|\det\mathrm{J}_{\mathbf{g}^{-1}}(\mathbf{y})|.$$ So say you grind through that computation, and you want to check that you did it right. How can you use the fact that you know how to sample $\mathbf{Y}$ to check that you correctly computed the density of $\mathbf{Y}$?
  • Say you are writing some software to implement a probability distribution (perhaps the one you derived above). So you want to write two functions: mydist_rand(...) for generating random draws, and mydist_pdf(...) for evaluating the pdf at a point. To test your software for errors, you want to check that the outputs of these two functions "agree." How can you do it?

(Update: Here is a related question with a great answer, but the proposed solutions seem applicable only to univariate random variables, and several assume that the CDF is easy to access, which I don't want to assume here.)

  • $\begingroup$ Are you familiar with chi-squared goodness-of-fit tests? If not, you can find out about them by searching our site. In higher dimensions the hardest part might consist of finding a convenient partitioning ("binning") of space that allows for adequately large expected counts in each bin and works well to detect the kinds of errors you might be worried about. $\endgroup$ – whuber Aug 28 '18 at 23:12

Essentially you need to calculate the distnace between the distributions.

You can consider specifically the supremum norm of the cumulative distribution $ Y_n = \max_{x} \left|\hat{F}_{n}\left(x\right) - F\left(x\right) \right|$, where $F$ is the cumulative distribution function calculated from the density, and $\hat{F}_{n}$ is the empirical commulative distribution function of $n$ samples.

Glivenko–Cantelli theorem states that $Y$ converges almost surely to zero, so if you want just to check yourself, this should be good enough. If you need a statistical test for this, you should consider Kolmogorov-Smirnoff test, which is based on Kolmogorov's theorem that provides the exact convergence rate.

Finally, note that since $F\left(x\right)$ is monotonically increasing, you can calculate $Y$ by examining the value of $F\left(x\right)$ only at the sampling points.

  • 1
    $\begingroup$ I agree in the univariate case, but I don’t think this extends easily to the multivariate case. $\endgroup$ – jcz Aug 29 '18 at 4:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.