I got the data, and plot the distribution of the data, and use the qqnorm function, but is seems doesn't follow a normal distribution, so which distribution should I use to discribe the data?

Empirical cumulative distribution function enter image description here

enter image description here

  • $\begingroup$ A plot of the histogram might also be useful to inspect. It might look like a skew-t distribution, rather than needing a mixture model. $\endgroup$ – John Jul 26 '12 at 18:28
  • 1
    $\begingroup$ Could you tell us a little about why you are trying to describe this distribution? After all, there are many alternatives to giving the name of some well known mathematical function: kernel smooths, n-letter summaries, the ECDF itself, and so on. All are adequate descriptions in appropriate settings. $\endgroup$ – whuber Jul 26 '12 at 20:11

I suggest you give heavy-tail Lambert W x F or skewed Lambert W x F distributions a try (disclaimer: I am the author). In R they are implemented in the LambertW package.

They arise from a parametric, non-linear transformation of a random variable (RV) $X \sim F$, to a heavy-tailed (skewed) version $Y \sim \text{Lambert W} \times F$. For $F$ being Gaussian, heavy-tail Lambert W x F reduces to Tukey's $h$ distribution. (I will here outline the heavy-tail version, the skewed one is analogous.)

They have one parameter $\delta \geq 0$ ($\gamma \in \mathbb{R}$ for skewed Lambert W x F) that regulates the degree of tail heaviness (skewness). Optionally, you can also choose different left and right heavy tails to achieve heavy-tails and asymmetry. It transforms a standard Normal $U \sim \mathcal{N}(0,1)$ to a Lambert W $\times$ Gaussian $Z$ by $$ Z = U \exp\left(\frac{\delta}{2} U^2\right) $$

If $\delta > 0$ $Z$ has heavier tails than $U$; for $\delta = 0$, $Z \equiv U$.

If you don't want to use the Gaussian as your baseline, you can create other Lambert W versions of your favorite distribution, e.g., t, uniform, gamma, exponential, beta, ... However, for your dataset a double heavy-tail Lambert W x Gaussian (or a skew Lambert W x t) distribution seem to be a good starting point.


### Set parameters ####
# skew Lambert W x t distribution with 
# (location, scale, df) = (0,1,3) and positive skew parameter gamma = 0.1
theta.st <- list(beta = c(0, 1, 3), gamma = 0.1)
# double heavy-tail Lambert W x Gaussian
# with (mu, sigma) = (0,1) and left delta=0.2; right delta = 0.4 (-> heavier on the right)
theta.hh <- list(beta = c(0, 1), delta = c(0.2, 0.4))

### Draw random sample ####
# skewed Lambert W x t
yy <- rLambertW(n=1000, distname="t", theta = theta.st)

# double heavy-tail Lambert W x Gaussian (= Tukey's hh)
zz =<- rLambertW(n=1000, distname = "normal", theta = theta.hh)

### Plot ecdf and qq-plot ####
op <- par(no.readonly=TRUE)
par(mfrow=c(2,2), mar=c(3,3,2,1))
qqnorm(yy); qqline(yy)

qqnorm(zz); qqline(zz)

ecdf and qqplot of skewed/heavy-tailed Lambert W x F distributions

In practice, of course, you have to estimate $\theta = (\beta, \delta)$, where $\beta$ is the parameter of your input distribution (e.g., $\beta = (\mu, \sigma)$ for a Gaussian, or $\beta = (c, s, \nu)$ for a $t$ distribution; see paper for details):

### Parameter estimation ####
mod.Lst <- MLE_LambertW(yy, distname="t", type="s")
mod.Lhh <- MLE_LambertW(zz, distname="normal", type="hh")

layout(matrix(1:2, ncol = 2))

enter image description here

Since this heavy-tail generation is based on a bijective transformations of RVs/data, you can remove heavy-tails from data and check if they are nice now, i.e., if they are Gaussian (and test it using Normality tests).

### Test goodness of fit ####
## test if 'symmetrized' data follows a Gaussian
xx <- get_input(mod.Lhh)

enter image description here

This worked pretty well for the simulated dataset. I suggest you give it a try and see if you can also Gaussianize() your data.

However, as @whuber pointed out, bimodality can be an issue here. So maybe you want to check in the transformed data (without the heavy-tails) what's going on with this bimodality and thus give you insights on how to model your (original) data.

  • $\begingroup$ +1 Very clear and nicely illustrated: you put some work into sharing these ideas with us and I thank you. $\endgroup$ – whuber Jan 17 '13 at 1:57
  • $\begingroup$ +1, thanks for posting this, I am interested in experimenting with this. Just a couple of minor clarifications: what does $\gamma \in \mathbb{R}$ refer to in your second paragraph? Also, in the sentence "In practice, of course, you have to estimate $\theta$", do you mean $\delta$? $\endgroup$ – Macro Jan 17 '13 at 2:42
  • 1
    $\begingroup$ @Macro: I made edits in the original post to clarify these two points. $\endgroup$ – Georg M. Goerg Jan 17 '13 at 3:03
  • $\begingroup$ I'm a fan, and started work on a scikit-learn like version here: github.com/gregversteeg/gaussianize $\endgroup$ – Greg Ver Steeg Apr 8 '16 at 2:35

This looks like an asymmetric distribution that has longer tails, in both directions, than the normal distribution.

  • You can see the long-tailedness because the observed points are more extreme than those expected under the normal distribution, on both the left and right side (i.e. they are below and above the line, respectively).

  • You can see the asymmetry because, in the right tail, the extent to which the points are more extreme than what would be expected under normal distribution is greater than it is in the left tail.

I can't think of any "canned" distributions that have this shape but it's not too hard to "cook up" a distribution that has the properties stated above.

Here is a simulated example (in R):

qqline(z) # see below for the plot. 
plot( ecdf(z) ) # see below for plot (2nd plot)

The variable here is a 50/50 mixture between an ${\rm exponential}(1)$ and an ${\rm exponential}(2)$ reflected around 0. This choice was made because it will be definitionally asymmetric, since there are different rate parameters, and they will both be long-tailed relative to the normal distribution, with the right tail being longer, since the rate on the right hand side is larger.

This example produces a pretty similar qqplot and empirical CDF (qualitatively) to what you're seeing:

enter image description here enter image description here

  • 1
    $\begingroup$ +1 Mixture models are useful. Especially if you have data that is generated under two or more different circumstances. $\endgroup$ – Seth Jul 26 '12 at 22:11
  • $\begingroup$ +1 The use of flexible distributions might be considered as well. $\endgroup$ – user10525 Jul 27 '12 at 13:00
  • $\begingroup$ @Procrastinator, what flexible distributions are you referring to (or is that a technical term)? $\endgroup$ – Macro Jul 27 '12 at 13:00
  • $\begingroup$ For example skew-t distributions: 1, 2, 3, 4, 5. They are unimodal, contain a skewness parameter and a kurtosis parameter, and avoid the assumption in mixture models that there are two or more populations generating the observations, as @Seth mentioned. $\endgroup$ – user10525 Jul 27 '12 at 13:05
  • 1
    $\begingroup$ @Macro That depends on how skewness is measured. The third moment is an ancient measure which is not quite accepted nowadays because it does not exist for many distributions such as the skew-t and symmetric $t$. The skew normal actually covers the whole range of other measures of skewness such as this one. Pearson's and Edgeworth's measures are influenced by the tails (because they are moment-based) which is an undesirable property. Also, they do not satisfy the 3 conditions required to be a measure of skewness specified in the paper linked ... $\endgroup$ – user10525 Jul 27 '12 at 13:21

In order to figure which distribution is the best fit, I would first identify some potential target distributions: I would think about the real world process that generated the data, then I would fit some potential densities to the data and compare their loglikelihood scores to see which potential distribution fit best. This is easy in R with the fitdistr function in the MASS library.

If your data is like Macro's z then:

[1] -2949.068

> fitdistr(z,'normal')$loglik
[1] -3026.648

> fitdistr(z,'t')$loglik
[1] -2830.861

So this gives the t distribution as best fitting (of those we tried) for Macro's data. confirm this with some qqplots using the parameters from fitdistr.

> qqplot(z,rt(length(z),df=2.7))  

Then compare this plot to the other distribution fits.

  • 1
    $\begingroup$ while I think fitting to known parametric families has its place, but what does that tell you in this case? From the asymmetry we know Cauchy, normal and $t$ can be ruled out right off of the bat. Can you think of any parametric families that could possibly capture this combination of asymmetry and long-tailedness? It seems to me that a mixture distribution (like the one I simulated from) may be needed, or a non-parametric estimate, as @whuber alludes to in his comment to the main question. $\endgroup$ – Macro Jul 26 '12 at 21:16
  • $\begingroup$ @Macro Many 'off the shelf' distributions can handle both skewed and heavy tailed situations. F and Gamma come to mind, along with nearly all 3 and 4 parameter distributions. I just added an answer so the original poster would have an idea about how to quantify the 'goodness of fit', and make numerical comparisons. $\endgroup$ – Seth Jul 26 '12 at 22:03
  • $\begingroup$ I get your point but I'm just trying to figure out whether any "off the shelf" distribution would work here. Gamma and F both are non-negative and I don't think Gamma can achieve this kind of shape, even if you shifted the data appropriately so that it was non-negative. $\endgroup$ – Macro Jul 26 '12 at 22:48
  • $\begingroup$ Seth and @Macro, the EDF plot in the question exhibits bimodality, so forget about achieving a good fit with a conventional distribution if there's a need to capture that second mode. At this time we don't have an effective criterion for recommending a fit. What if the OP has residuals from regressing flood data or financial catastrophes and therefore must get a good fit in the upper tail? What if it's essential for him/her to separate the modes? The answers will be strikingly different depending on the application. Distribution fitting has to be more than a blind mathematical exercise! $\endgroup$ – whuber Jul 27 '12 at 14:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.