12
$\begingroup$

I have the sample population of a certain signal's registered amplitude maxima. Population is about 15 million samples. I produced a histogram of the population, but cannot guess the distribution with such a histogram.

EDIT1: File with raw sample values is here: raw data

Can anyone help estimate the distribution with the following histogram: enter image description here

$\endgroup$
  • 1
    $\begingroup$ not that it matters dramatically but when using histograms it usually helps to have the relative frequency instead of absolute frequency on the y-axis. $\endgroup$ – posdef Mar 23 '11 at 10:27
  • $\begingroup$ that is, to provide 120000/15000000=0.008 instead 120000 on vertical axis? $\endgroup$ – mbaitoff Mar 23 '11 at 10:53
  • $\begingroup$ @mbaitoff: Your comments to the answer of schenectady indicate, that you are less interested in getting the name of the distribution but in finding out WHY the values are distributed this way. Is this correct ? $\endgroup$ – steffen Mar 23 '11 at 15:34
  • 1
    $\begingroup$ @mbaitoff, I'm not sure it would quite fit your application, but in related application areas, magnitudes of waves that undergo (many) random reflections between source and receiver are modeled by a Rayleigh distribution or one of its generalizations, e.g., Rice or Nakagami-$m$ distributions. $\endgroup$ – cardinal Mar 23 '11 at 20:01
  • 2
    $\begingroup$ The real interest in these data lies in the dozen or more spikes: the amount of data is large enough that those are real, in the sense that they are evidence of actual local modes. There seems to be a rich set of data here with a wealth of information that would be overlooked were a simple parametric formula used to summarize their distribution. $\endgroup$ – whuber Mar 25 '11 at 2:40
22
$\begingroup$

Use fitdistrplus:

Here's the CRAN link to fitdistrplus.

Here's the old vignette link for fitdistrplus.

If the vignette link doesn't work, do a search for "Use of the library fitdistrplus to specify a distribution from data".

The vignette does a good job of explaining how to use the package. You can look at how various distributions fit in a short period of time. It also produces a Cullen/Frey Diagram.

#Example from the vignette
library(fitdistrplus)
x1 <- c(6.4, 13.3, 4.1, 1.3, 14.1, 10.6, 9.9, 9.6, 15.3, 22.1, 13.4, 13.2, 8.4, 6.3, 8.9, 5.2, 10.9, 14.4)
plotdist(x1)
descdist(x1)

f1g <- fitdist(x1, "gamma")
plot(f1g)
summary(f1g)      

enter image description here

enter image description here

$\endgroup$
  • $\begingroup$ (+1): Didnt know that package before. $\endgroup$ – steffen Mar 23 '11 at 21:53
  • 1
    $\begingroup$ (+1( did not know that was called a Cullen/Frey diagram. I had to come up with that myself at one point. $\endgroup$ – Glen_b Apr 22 '13 at 6:09
  • $\begingroup$ the second image is with plotdist comamnd? How I can get the Cullen/Frey Diagram? $\endgroup$ – juanpablo Jul 2 '13 at 18:51
  • 1
    $\begingroup$ @juanpablo - Try descdist(). I updated the above post to include some code and a link to the old vignette. I could't get the above vignette link to work. So, Google the following: "Use of the library fitdistrplus to specify a distribution from data". It is a .pdf file. $\endgroup$ – bill_080 Jul 3 '13 at 0:10
  • 3
    $\begingroup$ @juanpablo - The statement f1g <- fitdist(x1, "gamma") fits a gamma distribution to the original data x1 and stores it in f1g. The upper left graph in plot(f1g) shows a histogram for the original data x1 as the bars, and the fitted gamma density plot from f1g as the continuous line. The density plot (continuous line) is drawn over the histogram as an indication of how well the "fit" represents the data. $\endgroup$ – bill_080 Jul 3 '13 at 15:13
5
$\begingroup$

Population is about 15 million samples.

Then you will very likely be able to reject any particular distribution of a simple, closed form.

Even that tiny bump at the left of the graph is likely to be enough to cause us to say 'clearly not such and such'.

On the other hand, it's probably pretty well approximated by a number of common distributions; obvious candidates are things like lognormal and gamma, but there are a host of others. It you look at the log of the x-variable, you can probably decide whether the lognormal is okay on sight (after taking logs, the histogram should look symmetric).

If the log is left skew, consider whether Gamma is okay, if it's right skew, consider whether inverse Gamma or (even more skew) inverse Gaussian is okay. But this exercise is more one of finding a distribution that's close enough to live with; none of these suggestions actually have all the features that appear to be present there.

If you have any theory at all to support a choice, toss out all this discussion and use that.

$\endgroup$
1
$\begingroup$

I am not sure why you would want to classify a sample to a specific distribution with such a large sample size; parsimony, comparing it to another sample, looking for physical interpretation of the paramters?

Most statistical packages(R, SAS, Minitab) allow one to plot data on a graph that yields a straight line if the data come from a particular distribution. I have seen graphs that yield a straight line if the data is normal(log normal-after a log transformation), Weibull, and chi-squared come to mine immediately. This technique will allow you to see outliers and give you the possiblity to assign reasons for why data points are outliers. In R, the normal probability plot is called qqnorm.

$\endgroup$
  • $\begingroup$ Good idea suggesting the qqplot. However, I think that your explanation of the technique is a little vague/hard to understand. Can you provide some exemplary R-code ? This would increase the value of the answer drastically. $\endgroup$ – steffen Mar 23 '11 at 12:07
  • $\begingroup$ I expect that somebody encountered the picture like mine and investigated the underlying distribution, because the values have physical basis. $\endgroup$ – mbaitoff Mar 23 '11 at 12:14
  • $\begingroup$ I am investigating the physical background of the sample distribution - how it is distributed and why. $\endgroup$ – mbaitoff Mar 23 '11 at 12:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.