Return to Answer

1. It is good to plot everything: functions, the random draws, empirical cumulative distributions against theoretical cumulative distribution functions etc. There is a number of plots that can and should be considered. This tremendously helps in finding bugs.

2. Check if $0 \le f(x) \le 1$ for discrete random variables and $0 \le f(x) < \infty$ for continuous random variables.

3. For discrete distributions $\left|1 - (\sum_x f(x))\right| \le \varepsilon$ for $x$'s in the support of $f$ ranging from some arbitrary small, up to arbitrary large value (for some arbitrary small $\varepsilon$) and $F(k) = \sum_k f(k)$.

4. Check if $0 \le F(x) \le 1$.

5. Check if $F(-\infty) = 0$ and $F(\infty) = 1$.

6. Check if $x = F^{-1}(F(x))$ or $p = F(F^{-1}(p))$

7. As noted by Xi'an, $F(X)$ should be uniformly distributed. Remember not to be very strict about uniformity of random draws from discrete distributions.

8. Moreover, in tests/p-r-random-tests.R R implements test based on an inequality of Massart:

   $$ \Pr\left( \sup |\hat F_n(x) - F(x)| > \lambda \right) \le 2\exp(-2n\lambda^2) $$

   where $\hat F_n(x)$ is the empirical distribution function, that can be used to compare the randomly generated samples to the cumulative distribution function. When using R, the code can be simply copy-and-pasted from R's source and re-used for testing.

9. It is important to check the boundary conditions, i.e. $x \in \{-\infty, 0, \infty\}$ especially since some distributions have discontinuities on $0$ that may be easily overseen (e.g. slash distribution). For bounded distribution it is important to check what happens on bounds and if $f(x) = 0$ for $x$ outside the support of $f$.

10. It is important to run checks on a wide range of parameter values (valid and invalid, e.g. against negative scale parameters).

11. It is good to have a deeper thought about handling invalid parameter values, missing data, NaN's etc. For example, base R propagates missing values and NaN's, it returns NaN's and throws warnings NaNs produced for invalid parameter values in the d/p/q functions and returns NA's and throws NAs produced warnings in r functions, etc.

12. Checking smoothness of the functions for very small (1e-13, 1e-14, 1e-15, 1e-16, ...), or very big values, may help you to diagnose the problems with numerical precision.

Some helpful hints are also given on slides Software for Distributions in R by David Scott, Diethelm Wurtz and Christine Dong.

1. It is a good idea to plot everything: functions, the random draws, empirical cumulative distributions against theoretical cumulative distribution functions etc. There is a number of plots that can and should be considered. This tremendously helps in finding bugs.

2. Check if $0 \le f(x) \le 1$ for discrete random variables and $0 \le f(x) < \infty$ for continuous random variables.

3. For discrete distributions $\left|1 - (\sum_x f(x))\right| \le \varepsilon$ for $x$'s in the support of $f$ ranging from some arbitrary small, up to arbitrary large value (for some arbitrary small $\varepsilon$) and $F(k) = \sum_k f(k)$.

4. Check if $0 \le F(x) \le 1$.

5. Check if $F(-\infty) = 0$ and $F(\infty) = 1$.

6. Check if $x = F^{-1}(F(x))$ or $p = F(F^{-1}(p))$

7. As noted by Xi'an, $F(X)$ should be uniformly distributed. Remember not to be very strict about uniformity of random draws from discrete distributions.

8. Moreover, in tests/p-r-random-tests.R R implements test based on an inequality of Massart:

   $$ \Pr\left( \sup |\hat F_n(x) - F(x)| > \lambda \right) \le 2\exp(-2n\lambda^2) $$

   where $\hat F_n(x)$ is the empirical distribution function, that can be used to compare the randomly generated samples to the cumulative distribution function. When using R, the code can be simply copy-and-pasted from R's source and re-used for testing.

9. Check the boundary conditions, i.e. $x \in \{-\infty, 0, \infty\}$ especially since some distributions have discontinuities on $0$ that may be easily overseen (e.g. slash distribution). For bounded distribution it is important to check what happens on bounds and if $f(x) = 0$ for $x$ outside the support of $f$.

10. Run checks on a wide range of parameter values (valid and invalid, e.g. against negative scale parameters).

11. It is good to have a deeper thought about handling invalid parameter values, missing data, NaN's etc. For example, base R propagates missing values and NaN's, it returns NaN's and throws warnings NaNs produced for invalid parameter values in the d/p/q functions and returns NA's and throws NAs produced warnings in r functions, etc.

12. Checking smoothness of the functions for very small (1e-13, 1e-14, 1e-15, 1e-16, ...), or very big values, may help you to diagnose the problems with numerical precision.

Some helpful hints are also given on slides Software for Distributions in R by David Scott, Diethelm Wurtz and Christine Dong.

After creating my own R package implementing a number of probability distributions, I have some thoughts about verifying correctness of the functions. For a nice starting point one could check the tests implemented in base R for testing the default distribution, that can be found in tests/d-p-q-r-tests.R and tests/p-r-random-tests.R files.

Moreover, it is important not to make equality checks when dealing with non-integers, since due to numerical precision, they will never be passed. Recall the advice given in Writing R Extensions document:

After creating my own R package implementing a number of probability distributions, I have some thoughts about verifying correctness of the functions. For a nice starting point one could check the tests implemented in base R for testing the default distribution, that can be found in tests/d-p-q-r-tests.R and tests/p-r-random-tests.R files.

Moreover, it is important not to make equality checks when dealing with non-integers, since due to numerical precision, they will never be passed. Recall the advice given in Writing R Extensions document:

1. It is good to plot everything: functions, the random draws, empirical cumulative distributions against theoretical cumulative distribution functions etc. There is a number of plots that can and should be considered. This tremendously helps in finding bugs.

2. Check if $0 \le f(x) \le 1$ for discrete random variables and $0 \le f(x) < \infty$ for continuous random variables.

3. For discrete distributions $\left|1 - (\sum_x f(x))\right| \le \varepsilon$ for $x$'s in the support of $f$ ranging from some arbitrary small, up to arbitrary large value (for some arbitrary small $\varepsilon$) and $F(k) = \sum_k f(k)$.

4. Check if $0 \le F(x) \le 1$.

5. Check if $F(-\infty) = 0$ and $F(\infty) = 1$.

6. Check if $x = F^{-1}(F(x))$ or $p = F(F^{-1}(p))$

7. As noted by Xi'an, $F(X)$ should be uniformly distributed. Remember not to be very strict about uniformity of random draws from discrete distributions.

8. Moreover, in tests/p-r-random-tests.R R implements test based on an inequality of Massart:

   $$ \Pr\left( \sup |\hat F_n(x) - F(x)| > \lambda \right) \le 2\exp(-2n\lambda^2) $$

   where $\hat F_n(x)$ is the empirical distribution function, that can be used to compare the randomly generated samples to the cumulative distribution function. When using R, the code can be simply copy-and-pasted from R's source and re-used for testing.

9. It is important to check the boundary conditions, i.e. $x \in \{-\infty, 0, \infty\}$ especially since some distributions have discontinuities on $0$ that may be easily overseen (e.g. slash distribution). For bounded distribution it is important to check what happens on bounds and if $f(x) = 0$ for $x$ outside the support of $f$.

10. It is important to run checks on a wide range of parameter values (valid and invalid, e.g. against negative scale parameters).

11. It is good to have a deeper thought about handling invalid parameter values, missing data, NaN's etc. For example, base R propagates missing values and NaN's, it returns NaN's and throws warnings NaNs produced for invalid parameter values in the d/p/q functions and returns NA's and throws NAs produced warnings in r functions, etc.

1. It is good to plot everything: functions, the random draws, empirical cumulative distributions against theoretical cumulative distribution functions etc. There is a number of plots that can and should be considered. This tremendously helps in finding bugs.

2. Check if $0 \le f(x) \le 1$ for discrete random variables and $0 \le f(x) < \infty$ for continuous random variables.

3. For discrete distributions $\left|1 - (\sum_x f(x))\right| \le \varepsilon$ for $x$'s in the support of $f$ ranging from some arbitrary small, up to arbitrary large value (for some arbitrary small $\varepsilon$) and $F(k) = \sum_k f(k)$.

4. Check if $0 \le F(x) \le 1$.

5. Check if $F(-\infty) = 0$ and $F(\infty) = 1$.

6. Check if $x = F^{-1}(F(x))$ or $p = F(F^{-1}(p))$

7. As noted by Xi'an, $F(X)$ should be uniformly distributed. Remember not to be very strict about uniformity of random draws from discrete distributions.

8. Moreover, in tests/p-r-random-tests.R R implements test based on an inequality of Massart:

   $$ \Pr\left( \sup |\hat F_n(x) - F(x)| > \lambda \right) \le 2\exp(-2n\lambda^2) $$

   where $\hat F_n(x)$ is the empirical distribution function, that can be used to compare the randomly generated samples to the cumulative distribution function. When using R, the code can be simply copy-and-pasted from R's source and re-used for testing.

9. It is important to check the boundary conditions, i.e. $x \in \{-\infty, 0, \infty\}$ especially since some distributions have discontinuities on $0$ that may be easily overseen (e.g. slash distribution). For bounded distribution it is important to check what happens on bounds and if $f(x) = 0$ for $x$ outside the support of $f$.

10. It is important to run checks on a wide range of parameter values (valid and invalid, e.g. against negative scale parameters).

11. It is good to have a deeper thought about handling invalid parameter values, missing data, NaN's etc. For example, base R propagates missing values and NaN's, it returns NaN's and throws warnings NaNs produced for invalid parameter values in the d/p/q functions and returns NA's and throws NAs produced warnings in r functions, etc.

12. Checking smoothness of the functions for very small (1e-13, 1e-14, 1e-15, 1e-16, ...), or very big values, may help you to diagnose the problems with numerical precision.