Suppose I have some data sitting in front of me, which is supposed to be sampled from a specific distribution. How would I go about testing whether it is from that distribution?

I've seen several tests which appear to do this - Pearson's chi-squared test, the one-sample Kolmogorov-Smirnov test, and the Anderson-Darling test, to name a few. Which one do I want here? (And if the answer is "it depends", then what does it depend on?)

  • 1
    $\begingroup$ I am affraid the answer to this is going to be a looong list of tests, model comparison techniques, goodness of fit tests, graphical tools, ... I believe a QQ plot is a good starting point in the univariate case. Actually, model selection in a more general scenario is one of the open problems in statistics proposed by Michael Jordan (the statistician). $\endgroup$
    – user10525
    Apr 27, 2012 at 16:18

1 Answer 1


There are many that can be used and the depends is that each look at different attributes of the distributions, so which is best depends on how you think your data is most likely to differ from the null distribution. One option would be to use several of them (to capture the different ways they might differ), but be sure to adjust for multiple comparisons. Also note that all the tests are rule out tests, if they are not significant that does not mean that the data comes from that distribution, just that you cannot rule it out. Also note that these tests often have low power to rule out many distributions at small sample sizes and with large sample sizes can find differences statistically significant that would not be considered practically significant.

One test that gives you more options to look at meaningful differences is the visual test described in:

Buja, A., Cook, D. Hofmann, H., Lawrence, M. Lee, E.-K., Swayne,
 D.F and Wickham, H. (2009) Statistical Inference for exploratory
 data analysis and model diagnostics Phil. Trans. R. Soc. A 2009
 367, 4361-4383 doi: 10.1098/rsta.2009.0120

Which consists of making several plots of data simulated from the null distribution along with the plot of the actual data, then see if someone not familiar with the data can tell which is the real data. This is implemented in the vis.test function in the TeachingDemos package for R or can be easily implemented using other tools.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.