6
$\begingroup$

The density of my data set was plotted in R as follows.

enter image description here

What kind of distribution would fit this data?

As I am not experienced to tell by visualization, I can only guess it is not normally distributed. I shall test it with R: some guidance as a starting point is highly desirable.

$\endgroup$
  • $\begingroup$ Does it has discrete or continuous values? $\endgroup$ – user88 Jul 26 '11 at 10:33
  • $\begingroup$ Does your data has negative values? $\endgroup$ – Dmitrij Celov Jul 26 '11 at 10:39
  • $\begingroup$ @Dimitrij I doubt it - the default for density in R assumes unbounded data hence the bleeding of density below zero. $\endgroup$ – Gavin Simpson Jul 26 '11 at 12:22
  • 1
    $\begingroup$ @Gavin, I doubt it myself, it looks like being from exponential or Gamma distribution, but more info would on the data source would be nice to know. $\endgroup$ – Dmitrij Celov Jul 26 '11 at 14:40
  • $\begingroup$ @Dmitrij, The values itself has no negative data. I know most of the model need to have negative data. Since I am only interested in the trend, not absolute value. I scale all values by a constant mean and a constant std dev. Then, I transformed all data using the equation of calculating Z-score, to make the number has both positive and negative value before fitting. Is it a reasonable approach? $\endgroup$ – evdstat Jul 27 '11 at 1:38
4
$\begingroup$

It looks rather like an exponential distribution (assuming that the bit below 0 is an artifact of smoothing in the density estimation).

I would look at a qqplot. In R, if x contains your data:

n <- length(x)
qqplot(x, qexp( (1:n - 0.5)/n ) )

Note that in the use of density() for the non-negative case, it is best to use from=0 since you know the density is 0 below 0.

plot(density(x, from=0))

I think also that, if $X$ follows an exponential distribution, then $e^{-X/\mu_X}$ should follow a uniform distribution, so the following could be a reasonable diagnostic:

hist(exp(-x/mean(x)), breaks=2*sqrt(length(x)))
| cite | improve this answer | |
$\endgroup$
3
$\begingroup$

It's not usually possible to identify a distribution from looking at a histogram like this.

As a start, plot the density on a log scale:

Log density plot

The tail of this density (from around 40 onward) is close to linear, showing it is close to exponential. That's part of the characterization. To go further, compare the density to this characterization by forming the residuals (on a log scale, effectively taking the ratio of the density to an exponential curve):

Residuals

Clearly this density is not exponential: for small values it is almost four times greater than the exponential fit to the tail would indicate. We must go further with the characterization.

We seek to characterize the residuals as simply as possible: this means in terms of longish straight segments or parabolic sections. (On this log scale, a straight segment is an exponential trend, whereas a parabolic section looks like a piece of a Normal distribution.) Evidently there are two parabolic-like sections: a sharp peaked one centered near 1 and a shallow, broad one centered near 25-30. The first would correspond to a healthy part of a truncated Normal distribution with small standard deviation (around 5-6) whereas the second would correspond to most of a Normal distribution with a larger standard deviation (around 10 perhaps). This indicates the density is not going to be adequately described by a simple mathematical formula, such as a Gamma or Weibull, but perhaps it can be decomposed into a mixture of two or three components. Look for each of those components to have some meaning: could these data indeed involve some combination of phenomena tending to occur near 1, near 25, and out beyond 40?

| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

Assuming, as others have, that the small blip below zero is an artifact of a density smoothing process, rather than a small amount of negative data, your distribution looks like an exponential distribution.

I'd start with either a exponential distribution, or the slightly more flexible Weibull distribution, and see if either one of those seems to fit well. Those two are a decent balance between difficulty to implement, visualize, etc. and having a decent likelihood of fitting your data.

| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

This is a long-tail distribution. GB2 (Generalized beta of second kind) with four parameters has a good flexibility for this kind of data. It's in package GB2.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.