9
$\begingroup$

Given a sample data set of floating point numbers, how do we determine its probability distribution and prove it? Also generate random numbers of the same distributions thereafter.

$\endgroup$
2
  • 2
    $\begingroup$ This question is perhaps overly broad. Do you have a specific problem you want to address? If so, please describe it and how you are generating the data, because those are crucial pieces of information for selecting probability distributions. $\endgroup$
    – whuber
    May 8, 2011 at 19:20
  • $\begingroup$ If all you have are the data, you can neither determine the distribution from which they were drawn, nor - even if you guessed it correctly - could you prove it. In some restricted situations, you can get somewhere. $\endgroup$
    – Glen_b
    Mar 24, 2016 at 1:31

3 Answers 3

14
$\begingroup$

The short answer is that you can't.

The longer answer is that you really need to think about what you are trying to accomplish and what question(s) you are trying to answer.

Tests on distributions are not designed to prove a particular distribution, but to disprove (they are not perfect for that, you still have type I and type II errors). But, often establishing an exact distribution is less important than finding a reasonable approximation. With round-off error and machine precision, you can not tell the difference between whether the data came from a normal distribution or another distribution that is only slightly different from the normal without an infinite amount of data (and still maybe not then due to the round-off). But treating such data as normal is probably still reasonable. The CLT tells us that we can often model using the normal distribution even when the data is clearly not from a normal distribution (provided we are modeling the behavior of the sample mean, not the population).

What is more important than the statistical tests and proofs is knowledge of the science that generated the data. Is a particular distribution (and the assumptions that go with it) reasonable from the science?

I prefer a visual test rather than the exact test for looking at distributions, generate data from the hypothesized distribution and create several plots, one with the original data, the rest with the generated data, then see if you can pick out which is different (the vis.test function in the TeachingDemos package for R does this). If you can't tell which is different then the hypothesized distribution is probably "close enough". Even if you can tell the difference you may decide that the differences are not that important.

If you want to generate new data from a distribution similar to your existing data then you can take bootstrap samples, or bootstrap samples plus some random noise (this is sampling from the kernel density estimate), or you can do a logspline fit and generate from that distribution (see the logspline package for R as one tool for this).

$\endgroup$
6
$\begingroup$

The only way to "prove" that data comes from a certain distribution (without an infinite number of samples) is to know precisely how that data is generated. For example, if you know that the data came from the magnitude of a circular bivariate normal random variable, it has a Rician distribution. Or if the data came from the time between events in a Poisson process, then it has an exponential distribution.

Lacking a precise definition of the generating process, there are a number of empirical measures you can use to determine the underlying distribution. First, look at the data itself: Is it discrete or continuous? Is it supported on (-inf,inf), [0,inf),(0,1), or another interval? This knowledge can be used to narrow down the possible univariate parametric distributions that could fit your data. Examples include the Gaussian distribution, Cauchy, Exponential, Gamma, Generalized Extreme Value, Rician, Wrapped Cauchy, Von Mises, Binomial, and Beta. Once you have determined the support of the distribution, test the potential univariate distributions with an information criterion - such as Akaike information criterion (AIC) or Bayesian information criterion (BIC). These balance the number of parameters in a given distribution with the likelihood of the data fitting a given distribution. Visually check the best-scoring distribution(s) to see if they appear to fit the data.

An alternative is to construct a kernel density estimate of the data. This is basically a sophisticated version of creating a histogram of the data, where a small Gaussian (or other) distribution is placed at each data point, and the estimated distribution is constructed from the sum of these. For more information, see Kernel Density Estimation. This has the advantage of being able to fit arbitrary distributions in the data, but sampling from this distribution has a large computational cost, especially with large data sets.

Another option is to construct a Gaussian Mixture Model (GMM) from the data, where a small number of Gaussian distributions are used to approximate the underlying distribution. For more information, see Mixture Models.

The method appropriate for your application depends on the application itself. If you can determine the distribution from the generating process, great, estimate the parameters and your done! If not, the next-best scenario is finding a univariate parametric distribution that accurately describes the data. Lacking that, mixture models, KDEs, or other methods can be used to approximate the distribution.

$\endgroup$
2
$\begingroup$

If you're trying to do Exploratory Data Analysis, you could use some graphical techniques.

I can suggest chapter 1.3.4 in the NIST handbook. In particular, any of the probability graphs might be insightful (e.g. Probability Plot Correlation Coefficient Plot, Quantile-Quantile Plot, etc.).

For a number of common distributions, you could try fitting to the Tukey-Lambda distribution, and extracting the distribution information from the fitted value of the lambda shape parameter.

$\endgroup$
3
  • $\begingroup$ Also check out the worked example there. (not enough rep. to add this link in the original answer...)-; $\endgroup$
    – Inon
    Mar 22, 2016 at 18:31
  • $\begingroup$ (+1) These recommendations look sufficiently constructive to be posted as an answer, especially since (as has been remarked in other answers) the request to "prove it" is misguided. Thank you for visiting our site and contributing! $\endgroup$
    – whuber
    Mar 22, 2016 at 19:29
  • $\begingroup$ Thanks, @whuber. I guess some visitors put more emphasis on a theoretical approach to statistics than addressing more pragmatic needs. That might even be a decent 'meta' post... $\endgroup$
    – Inon
    Mar 23, 2016 at 19:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.