Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
The density of my data set was plotted in R as follows.
What kind of distribution would fit this data?
As I am not experienced to tell by visualization, I can only guess it is not normally distributed. I shall test it with R: some guidance as a starting point is highly desirable.
It looks rather like an exponential distribution (assuming that the bit below 0 is an artifact of smoothing in the density estimation).
I would look at a qqplot. In R, if x contains your data:
n <- length(x)
qqplot(x, qexp( (1:n - 0.5)/n ) )
Note that in the use of density() for the non-negative case, it is best to use from=0 since you know the density is 0 below 0.
plot(density(x, from=0))
I think also that, if $X$ follows an exponential distribution, then $e^{-X/\mu_X}$ should follow a uniform distribution, so the following could be a reasonable diagnostic:
hist(exp(-x/mean(x)), breaks=2*sqrt(length(x)))
It's not usually possible to identify a distribution from looking at a histogram like this.
As a start, plot the density on a log scale:
The tail of this density (from around 40 onward) is close to linear, showing it is close to exponential. That's part of the characterization. To go further, compare the density to this characterization by forming the residuals (on a log scale, effectively taking the ratio of the density to an exponential curve):
Clearly this density is not exponential: for small values it is almost four times greater than the exponential fit to the tail would indicate. We must go further with the characterization.
We seek to characterize the residuals as simply as possible: this means in terms of longish straight segments or parabolic sections. (On this log scale, a straight segment is an exponential trend, whereas a parabolic section looks like a piece of a Normal distribution.) Evidently there are two parabolic-like sections: a sharp peaked one centered near 1 and a shallow, broad one centered near 25-30. The first would correspond to a healthy part of a truncated Normal distribution with small standard deviation (around 5-6) whereas the second would correspond to most of a Normal distribution with a larger standard deviation (around 10 perhaps). This indicates the density is not going to be adequately described by a simple mathematical formula, such as a Gamma or Weibull, but perhaps it can be decomposed into a mixture of two or three components. Look for each of those components to have some meaning: could these data indeed involve some combination of phenomena tending to occur near 1, near 25, and out beyond 40?
Assuming, as others have, that the small blip below zero is an artifact of a density smoothing process, rather than a small amount of negative data, your distribution looks like an exponential distribution.
I'd start with either a exponential distribution, or the slightly more flexible Weibull distribution, and see if either one of those seems to fit well. Those two are a decent balance between difficulty to implement, visualize, etc. and having a decent likelihood of fitting your data.
This is a long-tail distribution.
GB2 (Generalized beta of second kind) with four parameters has a good flexibility for this kind of data.
It's in package GB2.
densityin R assumes unbounded data hence the bleeding of density below zero. $\endgroup$ – Gavin Simpson Jul 26 '11 at 12:22