I have data on the first to fourth moments of a continuous random variable and I am trying to find what distribution best fits the data. Wikipedia has a list of about 20 distributions that could fit the data and I would like to try each of them to see which is the best. Some of these distributions have one parameter, some have two or three. Is there some test I can use to compare the fit of each distribution? Or at least a test to compare distributions with the same number of parameters?
$\begingroup$
$\endgroup$
8
-
$\begingroup$ (1) Moments, even when measured without sampling error, don't determine distributions particularly well; sometimes even an infinite number of moments is insufficient. (2) Why would the list of distributions in Wikipedia happen to coincide with your data? Do you know/assume anything that would cut the problem down (boundedness, unimodality, continuously differentiable, ...)? $\endgroup$– Glen_bCommented Apr 26, 2014 at 0:45
-
$\begingroup$ @Glen_b Unfortunately all I know is that the variable can take any value on the real line and that by definition its mean is zero. It is not a property that is well understood. But thank you for your suggestion, I had not thought about exotic properties like differentiability. I cannot derive the distribution but I would expect it to be something that arises in nature so I think wikipedia would have it in its list. $\endgroup$– HughCommented Apr 26, 2014 at 1:49
-
1$\begingroup$ "Something that arises in nature" and "something that wikipedia has in its list" are likely to be very close to disjoint sets. $\endgroup$– Glen_bCommented Apr 26, 2014 at 3:16
-
1$\begingroup$ Things that arise in nature are normally studied, things studied are often on wikipedia. Anyway since I don't have a way to decide what the distribution is all I can do is try to fit data to many distributions and wiki seems to have a comprehensive list. As this is for an engineering project it does not matter if I don't have the exact distribution, only if the distribution causes results which are expected. Two distributions could both be suitable. $\endgroup$– HughCommented Apr 26, 2014 at 7:17
-
$\begingroup$ I accept both premises in your first sentence. However, the conlcusion that the probability distributions that are on Wikipedia are therefore found in nature does not follow from those premises. When you say "only if the distribution causes results which are expected" we're getting closer to a good criterion, as long as we can pin down what you mean by 'results which are expected' -- what aspects of the distribution matter? $\endgroup$– Glen_bCommented Apr 26, 2014 at 7:37
|
Show 3 more comments
4 Answers
$\begingroup$
$\endgroup$
2
There is an extensive derivation and discussion of the Pearson family in Chapter 5 of our book:
- Rose and Smith, "Mathematical Statistics with Mathematica", Springer, NY
You can download the chapter for free here:
http://www.mathstatica.com/book/Rose_and_Smith_2002edition_Chapter5.pdf
See Section 5.2 D (p.159 on) for a derivation of Pearson Coefficients in terms of the first 4 population moments. [There is also a fun derivation in terms of the first 6 moments, which may well have been a first at the time.] The Pearson system generally works rather nicely if the skewness and kurtosis of the data are not tooooo large ... it is best to check to see where $\beta_1$ and $\beta_2$ lie on the Pearson diagram as a first step. If, however, skewness and/or kurtosis are 'large', and you are working with sample data rather than population data, then the estimates of the moments can become unreliable, and the method is less useful.
P.S. If you would like to add your moment data to the question (specifying if they are raw moments or central moments), I'd be happy to generate the Pearson fit for you. Only takes a few seconds to do. Ideally, please specify the central moments $\mu_2$, $\mu_3$ and $\mu_4$ and the mean ... but if you only have raw moments, that should be fine too.
-
1$\begingroup$ Thank you very much for the link to the book, I am actually planning a thesis at the moment so I don't have data yet. Is there a program you use the generate the fit in seconds? I am able to use R and Matlab. $\endgroup$– HughCommented Apr 26, 2014 at 1:54
-
2$\begingroup$ @user27271 Yes - the software used is discussed in the above chapter. It runs with Mathematica. $\endgroup$– wolfiesCommented Apr 26, 2014 at 6:51
$\begingroup$
$\endgroup$
1
Try Hahn and Shapiro Statistical Models in Engineering and the discussion of the Pearson Distribution. This is illustrated within Wikipedia and relationships to well-known distributions are shown in section 4 of the same page. You can also find more background information here.
-
$\begingroup$ @user27271 - Following the link above you could prepare a simple select case procedure that would accept your first 4 moments, compute B1 and B2, and identify a potential distribution. This could be combined with a histogram for inspection purposes. It would be fairly straightforward to do this in R. $\endgroup$– AsymLabsCommented Apr 26, 2014 at 15:06
$\begingroup$
$\endgroup$
I would suggest maximum entropy density estimation, but I could not find an off-the-shelf application to do it. However, you may find this paper useful: A Computationally Efficient Multivariate Maximum-Entropy Density Estimation (MEDE) Technique.
$\begingroup$
$\endgroup$
I like your question! But I think even when using 20 or more distributions you can fail easily. If your data has e.g. multiple modes, then a Gaussian mix maybe of 2 or even more Gaussians could fit well, but also mixes of other distributions like log-norm or Student T. To decide which is best you will really need a lot of sample points like N>>1000. Consider to exploit also other information, like or all samples >0, or are there sharp bounds (like uniform) or long tails (often leading to non-existing high-order moments!)?? Maybe you know more about your design, like the key variables have Gaussian distribution, but transfer function is a polynomial (or exponential or rational or whatever)?
