28
$\begingroup$

I am a grad student developing an interest for statistics. I like the material over-all, but I sometimes have a hard time thinking about applications to real life. Specifically, my question is about commonly used statistical distributions (normal - beta- gamma etc.). I guess for some cases I get the particular properties that make the distribution quite nice - memoryless property of exponential for example. But for many other cases, I don't have an intuition about both the importance and application areas of the common distributions that we see in textbooks.

There are probably a lot of good sources addressing my concerns, I would be glad if you could share those. I would be a lot more motivated into the material if I could associate it with real-life examples.

$\endgroup$
23
$\begingroup$

Wikipedia has a page that lists many probability distributions with links to more detail about each distribution. You can look through the list and follow the links to get a better feel for the types of applications that the different distributions are commonly used for.

Just remember that these distributions are used to model reality and as Box said: "all models are wrong, some models are useful".

Here are some of the common distributions and some of the reasons that they are useful:

Normal: This is useful for looking at means and other linear combinations (e.g. regression coefficients) because of the CLT. Related to that is if something is known to arise due to additive effects of many different small causes then the normal may be a reasonable distribution: for example, many biological measures are the result of multiple genes and multiple environmental factors and therefor are often approximately normal.

Gamma: Right skewed and useful for things with a natural minimum at 0. Commonly used for elapsed times and some financial variables.

Exponential: special case of the Gamma. It is memoryless and scales easily.

Chi-squared ($\chi^2$): special case of the Gamma. Arises as sum of squared normal variables (so used for variances).

Beta: Defined between 0 and 1 (but could be transformed to be between other values), useful for proportions or other quantities that must be between 0 and 1.

Binomial: How many "successes" out of a given number of independent trials with same probability of "success".

Poisson: Common for counts. Nice properties that if the number of events in a period of time or area follows a Poisson, then the number in twice the time or area still follows the Poisson (with twice the mean): this works for adding Poissons or scaling with values other than 2.

Note that if events occur over time and the time between occurrences follows an exponential then the number that occur in a time period follows a Poisson.

Negative Binomial: Counts with minimum 0 (or other value depending on which version) and no upper bound. Conceptually it is the number of "failures" before k "successes". The negative binomial is also a mixture of Poisson variables whose means come from a gamma distribution.

Geometric: special case for negative binomial where it is the number of "failures" before the 1st "success". If you truncate (round down) an exponential variable to make it discrete, the result is geometric.

$\endgroup$
  • 3
    $\begingroup$ Well thank you for your answer. However, wikipedia provides a more general description that I'd like. Basically, my question is why some distributions are nice? To give a possible answer in case of normal distribution, may be related to central limited theorem - which says if you sample an infinite amount of observations you can actually in asympotics see that the a sufficient statistic of those observations, given independence have a normal distribution. I am searching for more examples like that.. $\endgroup$ – Roark Aug 6 '12 at 21:59
  • $\begingroup$ Not exactly a real distribution, but what about bimodal? I can't think of any commonly seen real-life examples after I found that the many of the gender differences in human are not bimodal. $\endgroup$ – ceiling cat Jun 29 '15 at 19:46
  • $\begingroup$ Add multinomial $\endgroup$ – user46925 Mar 10 '16 at 20:19
3
$\begingroup$

Asymptotic theory leads to the normal distribution, the extreme value types, the stable laws and the Poisson. The exponential and the Weibull tend to come up as parametric time to event distributions. In the case of the Weibull it is an extreme value type for the minimum of a sample. Related to the parametric models for normally distributed observations the chi square, t and F distributions arise in hypothesis testing and confidence interval estimation.The chi square also come up in contingency table analysis and goodness of fit tests. For studying power of tests we have the noncentral t and F distributions. The hypergeometric distribution arises in Fisher's exact test for contingency tables. The binomial distribution is important when doing experiments to estimate proportions. The negative binomial is an important distribution to model overdispersion in a point process. That should give you a good start on pratical parametric distrbutions. For nonnegative random variables on (0, ∞) the Gamma distribution is flexible for providing a variety of shapes and the log normal is also commonly used. On [0,1] the beta family provides symmetric distirbutions including the uniform as well as distributions skewed left or skewed right.

I should also mention that if you want to know all the nitty gritty details about distributions in statistics there are the classic series of books by Johnson and Kotz that include discrete distributions, continuous univariate distributions and continuous multivariate distributions and also volume 1 of the Advanced Theory of Statistics by Kendall and Stuart.

$\endgroup$
  • $\begingroup$ Thank you vey much for the answer, this is extremely useful. Thank you again, it really helped me. $\endgroup$ – Roark Aug 7 '12 at 4:56
2
$\begingroup$

Buy and read at least the first 6 chapters (first 218 pages) of William J. Feller "An Introduction to Probability Theory and Its Applications, Vol. 2" http://www.amazon.com/dp/0471257095/ref=rdr_ext_tmb . At least read all of the Problems for Solution, and preferably try solving as many as you can. You don't need to have read Vol 1, which in my opinion is not particularly meritorious.

Despite the author having died 45 1/2 years ago, before the book was even finished, this is simply the finest book there is, bar none, for developing an intuition in probability and stochastic processes, and understanding and developing a feel for various distributions, how they relate to real world phenomena, and various stochastic phenomena which can and do occur. And with the solid foundation you will build from it, you will be well served in statistics.

If you can make it though subsequent chapters, which gets somewhat more difficult, you will be light years ahead of almost everyone. Simply put, if you know Feller Vol 2, you know probability (and stochastic processes); meaning that, anything you don't know, such as new developments, you will be able to quickly pick up and master by building on that solid foundation.

Almost everything previously mentioned in this thread is in Feller Vol 2 (not all of the material in Kendall Advanced Theory of Statistics, but reading that book will be a piece of cake after Feller Vol 2), and more, much more, all of it in a way which should develop your stochastic thinking and intuition. Johnson and Kotz is good for minutiae on various probability distributions, Feller Vol 2 is useful for learning how to think probabilistically, and knowing what to extract from Johnson and Kotz and how to use it.

$\endgroup$
2
$\begingroup$

Just to add to the other excellent answers.

The Poisson distribution is useful whenever we have counting variables, as others have mentioned. But much more should be said! The poisson arises asymptotically from a binomially distributed variable, when $n$ (the number of Bernoulli experiments) increases without bounds, and $p$ (the success probability of each individual experiment() goes to zero, in such a way that $\lambda=n p$ stays constant, bounded away from zero and infinity. This tells us that it is useful whenever we have a large number of individually very improbable events. Some good examples are: accidents , such as number of car crashes in New York in a day, since each time two cars passes/meets there are a very low probability of a crash, and the number of such opportunities is indeed astronomical! Now you yourself can think about other examples, such as total number of plane crashes in the world in a year. The classic example where the number of deaths by horsekicks in the Preussian cavalry!

When the Poisson is used in epidemiology, for modelling number of cases of some sickness, one often finds it does not fit well: The variance is too large! The Poisson has variance=mean, which can be seen easily from the limit of binomial: In the binomial the variance is $n p (1-p)$, and when $p$ goes to zero necessarily $1-p$ goes to one, so the variance goes to $np$, which is the expectation, and those both go to $\lambda$. One way is to search for an alternative to the Poisson with larger variance, not conditioned to equal the mean, such as the negative binomial. ¿But why do this phenomenon of larger variance, occur? One possibility is that the individual probabilities of sickness $p$ for one person, are not constant, and neither depends on some observed covariate (say age, occupation, smoking status, ...) That is called unobserved heterogeneity, and sometimes models used for is is called frailty models, or mixed models. One way of doing this is assuming the $p$'s in the population comes from some distribution, and assuming that is a gamma distribution, for instance (which makes for simpler maths...), we get the gamma-poisson distribution --- which recovers the negative binomial!

$\endgroup$
0
$\begingroup$

Recently published research suggests that human performance is NOT normally distributed, contrary to common thought. Data from four fields were analyzed: (1) Academics in 50 disciplines, based on publishing frequency in the most pre-eminent discipline-specific journals. (2) Entertainers, such as actors, musicians and writers, and the number of prestigious awards, nominations or distinctions received. (3) Politicians in 10 nations and election/re-election results. (4) Collegiate and professional athletes looking at the most individualized measures available, such as the number of home runs, receptions in team sports and total wins in individual sports. The author writes, "We saw a clear and consistent power-law distribution unfold in each study, regardless of how narrowly or broadly we analyzed the data..."

$\endgroup$
  • 4
    $\begingroup$ Who suggested that human performance is normally distributed?! The 80-20 principle was suggested by Pareto (1906!). $\endgroup$ – abaumann Mar 8 '14 at 11:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.