Pedagogical order of study for named distributions

Question

I've seen myriads of named probability densities or distributions in multiple books and courses, usually both Binomial and Bernoulli are among the first discrete ones, while for continuous they use Normal and Poisson as examples. Following this train of thought, I wondered if there were any "fundamental" distributions. I found this link to contain a useful graphical summary of the many distributions out there and how they relate to each other. Judging by the amount of arrows going in and out, the most 'important' ones appear to be normal, exponential, binomial, chi squared.

However, from a pedagogical standpoint and based on how conceptually valuable they can be on posterior study I ask for a sensible "study order" of important distributions, since I also know that some distributions "approach" or approximate" others when the samples increase or in an appropriately taken limit.

Answer 1

Any answer will be a matter of opinion, but I have taught lots of courses using what I believe to be a fairly standard order of presentation. I think there are good reasons for it and will discuss some of them.

Many elementary probability courses start with empirical discrete distributions. Proportions of various colors of candy bits in a large package of M&M's Skittles, or whatever. Some combinatorial material provides a basis for a few elementary distributions. Some basic rules of of probability are discussed.

After that and a discussion of tossing fair coins it is natural to get into a discussion of binomial distributions, which can involve some of the combinatorial arguments and probability rules. Also, a proof or discussion of the Law of Large numbers often appears at this point.

Next, depending on the level and applied nature of the course, it may be natural to discuss Poisson distributions as a limiting case of binomial one, and geometric and negative binomial distributions as waiting times for events already discussed in a binomial context. and to consider hypergeometric distributions as a generalization of binomial distributions where trials are not independent.

At some point a bridge needs to be crossed to discuss how continuous distributions differ fundamentally from discrete ones. Often starting with a brief mention of uniform distributions because of their mathematical simplicity, it is customary to spend a lot of time on normal distributions, because of their widespread use in applications, and because the Central Limit Theorem shows (or illustrates) convergence to normal.

Then a discussion of using normal distributions to approximate binomial probabilities seems mandatory. (At this point it seems especially worthwhile nowadays to show how better results are available from statistical software.)

Next, some courses introduce exponential distributions, which are widely used in applications and are in many respects simpler to handle than normal distributions. That can lead to a discussion of other gamma family distributions. If the course has any kind of Bayesian flavor, beta distributions are often discussed as natural prior distributions for binomial success probabilities.

The specific distributions discussed later in the course depend on the purpose of the course in a theoretical or applied program. It is not possible in a single course to deal with all of the distributions and relationships among them that are displayed on your link. From there on there seems to be no traditional order as various objectives are pursued.

Answer 2

You might want to distinguish between

1. distributions of random variables in general and

2. distributions of a special class of random variables, namely test statistics like the sum of random variables. Such distributions are called sampling distributions. These distributions are of special importance when talking about statistical tests and thus you might call them "pedagogically" important.

Examples for 2.

• The normal distribution. The sum/average of i.i.d. normal random variables is again normal and many important test statistics like e.g. regression coefficients tend to look normal for large samples.

• The F distribution: In ANOVA settings, normalized ratios of variances are of special importance. Under strict assumptions, such ratios follow an F distribution.

• The Chi-squared distribution: It serves as an approximation to the F distribution.

• The Binomial distribution: Is is of key importance wherever one studies properties of the sum of i.i.d. Bernoulli random variables.

Examples for 1.

• Money and time tend to be right skewed. Models for right skewed random variables include the exponential, the gamma, the Weibull distribution, amonst many others.

• The normal and the logistic distribution are models for bell-shaped distributions.

• The Bernoulli distribution is the (only) model for binary random variables.

• The multinomial distribution describes the distribution of a discrete random variable with finite number of values.

• The Poisson, Negative Binomial are models for counts.

• The Beta distribution serves as model for continuous random variables squeezed between 0 and 1.

• The uniform distribution ...

Answer 3

If you want to teach the concept of a distribution, I think it's best to start with the simplest ones, which are probably the uniform for both continuous and categorical data.

The equations behind the uniform are straightforward and therefore don't get in the way of teaching the concepts, which are going to be new to people, whether you are in a mathematical sequence (i.e. in a math department with students having had at least a couple semesters of calculus) or a practical sequence (i.e. not in a math department, with students who may not even have had two years of high school algebra).

Answer 4

Thinking back at the first course when I learned probability theory (OK, I had been doing some independent reading at high school), we used this book by Kai Lai Chung, and "named distributions" was not a topic at all. Focus was on probabilistic principles, and there we saw what I later learnt was called Bernoulli, Binomial, geometric, negative binomial which all occurs in natural elementary problems. Later occurred Poisson and exponential (natural example as waiting time in Poisson processes) and even the normal distribution since it was needed for the central limit theorem ...

I still find that a good approach, a first introduction must focus on principles and techniques, grand ideas like coin tossing, natural examples ... and not on some library of "named distributions." One big advantage is that students must think from first principles, they cannot just ask themselves "which distribution is this?"

Answer 5

Thanks for the link. Usually one starts teaching with the normal distribution. But, just look at the diagram that you linked to and count the number of linkages to each distribution. That indicates which are more related as points of departure for other, less commonly explored, distributions, and which have been, as an historical matter, related as more central or focal to the others. Unfortunately, no chart of the type that you linked to is exhaustive because there are many undocumented distributions, or those which are seldom used or little known, for example, the gamma-Pareto distributions, and a plethora of other distributions that are convolutions like the gamma convolution distribution, the Pareto convolution distribution or even the Pearson distribution family, none of which appear in the chart.

Where to begin also is heavily influenced by who the students are and what they need to know. For example, physics students who must learn quantum mechanics have very different needs than business school freshmen. Statisticians may want the rules for creating distributions, and mathematicians might want to start with the moment generating function. So, different flavors for different folks.