Given a sample of $N$ observed values I'd like to test the null hypothesis that they arose from an arbitrary PDF (for which I have the analytical form). There are tests in place that can handle some of the well known PDFs (i.e., normal and such) but I've found no way (or package) to extend this process to an arbitrary PDF.

Currently what I do is the following:

  1. Given the PDF, construct its CDF
  2. Invert the CDF (numerically if necessary)
  3. Sample $N$ random uniform values in the range $[0, 1]$
  4. Obtain the PDF values evaluating these $N$ random uniform values in the inverted CDF
  5. Use the Anderson-Darling k-sample test to test the hypothesis that both samples (original and sampled) originated from the same distribution

This works, but I'd like a more direct approach. Is it possible? How would one do that?

  • $\begingroup$ Do you have alternative distributions that it could have arisen from or is your interest strictly did they arise from distribution $\mathcal{f}$. $\endgroup$ – Dave Harris Apr 8 '20 at 14:35
  • $\begingroup$ No, I'm just interested in the hypothesis that they arose from the $f$ distribution. $\endgroup$ – Gabriel Apr 8 '20 at 14:39

Could you use the Kolmogorov-Smirnov test to compare your distributions? From Wikipedia,

"The Kolmogorov–Smirnov statistic quantifies a distance between the empirical distribution function of the sample and the cumulative distribution function of the reference distribution, or between the empirical distribution functions of two samples... In the one-sample case, the distribution considered under the null hypothesis may be continuous, purely discrete or mixed. In the two-sample case, the distribution considered under the null hypothesis is a continuous distribution but is otherwise unrestricted."

It's pretty flexible and well-implemented. If you know the form of your PDF, you could obtain the respective CDF form and then use a one-sample KS test to compare an empirical distribution to that CDF.

  • $\begingroup$ I usually prefer the AD over the KS test as per Beware the Kolmogorov-Smirnov test! but this sounds like a reasonable approach (also the AD implementation in scipy does not seem to be able to handle arbitrary CDFs) $\endgroup$ – Gabriel Apr 8 '20 at 14:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.