Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have a data set of 100,000+ event times. For this study an event begins (at time 0) and can run for an indeterminate amount of time. The bulk of events require a couple of hours to complete but there are also events that can take tens of hours to complete. I created a histogram of the times and it resembles a positively skewed distribution.

My question is: how can I determine a distribution type to model this data? For example, the data looks log-normal, but a histogram of the log of the data is not a normal distribution (it is a negatively skewed distribution). I spent some time looking at the Gamma distribution but besides visually resembling my data I don't know if Gamma is an appropriate selection. I also had a look at the type-2 Gumbel distribution but only because it is asymmetric and semi-infinite, like my data. What are the rules for picking a distribution to model a data set?

share|improve this question
    
This scenario might be related to a Poisson distribution. It models the probability of events occurring in a certain interval. In any event you might want to specify your hypothesis here. Potentially you could answer your question with a non-parametric test, in which case you would not need to assume a certain distribution. –  user12719 Dec 3 '12 at 16:08
add comment

3 Answers

up vote -1 down vote accepted

You will need to iterate through all of the know theoretical distributions to find the correct distribution to use. In Windows, you may use ExpertFit or EasyFit for Excel.

If you are comfortable with R, you can find the minimum log-likelihood function of each distribution.

http://stackoverflow.com/questions/4290081/fitting-data-to-distributions

The Kolmogorov-Smirnov, Anderson-Darling, Chi-Squared Goodness-of-Fit Tests should be able to help you.

http://www.itl.nist.gov/div898/handbook/eda/section3/eda35g.htm

Let me know if you need anything else.

share|improve this answer
    
Thanks for the link to the post about fitdistr. I will use that to determine what distribution best fits my data. –  Joseph Ryan Glover Dec 4 '12 at 13:55
add comment

Waiting times are often modeled using the Exponential distribution (note the section on memorylessness) or the Weibull distribution, which is a generalization of the Exponential.

share|improve this answer
    
As per my response above to @dpott197 I am going to use R's fitdistr functionality to try several distributions against my data. I will try both the exponential and Weibull distributions to see how they fit. Thanks for the suggestion. –  Joseph Ryan Glover Dec 4 '12 at 13:56
add comment

A Gamma distribution is highly likely to be a good choice, but I would start with the Exponential distribution which is a special case of the Gamma. The exponential distribution is the default starting point for distributions of the time waiting for an event to occur.

I would strongly recommend graphical methods as a starting point - the Quantile - Quantile plot is the stanard tool for comparing one distribution with another (eg your empirical observed distribution with a theoretical one).

share|improve this answer
    
As per my response to @dpott197 I am going to use R's fitdistr functionality to test my data against several distributions, one of which will be Gamma. Thank you for the tip about the Q-Q plot. –  Joseph Ryan Glover Dec 4 '12 at 13:57
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.