# How do statisticians determine which distribution is appropriate for different statistical tests?

For example, the test statistic calculated for the ANOVA test is compared to an F-distribution, while a t-test comparing means compares the test statistic to a t-distribution.

• For a general overview, have a look at page 3 of this paper. It contains a chart depicting the relationships between many distributions. Quite neat. – COOLSerdash Sep 23 '13 at 19:22
• At one level, the answer is simple: the distribution is that of the test statistic under the null hypothesis. Finding it is merely a calculation. The hard parts are coming up with a suitable probability model for a problem, eliciting a loss function, and finding a test statistic that produces a good test. Many distributions, including the Normal, $t$, and $\chi^2$, actually appear most often as asymptotic approximations to the actual distributions (and therein lies a separate part of any good answer). – whuber Sep 24 '13 at 22:02

The full answer to your question would be a full semester math-theory statistics course (which would be a good idea for you to take if you are really interested).

But a short and partial set of answers are:

Generally we start with the normal distribution, it has been found to be a reasonable approximation for many real world situations and the Central Limit theorem (and others) tell us that it is an even better approximation when looking at the means of simple random samples (bigger sample size leads to better approximation by the normal). So the normal is often the default distribution to consider if there is not a reason to believe that it will not be a reasonable approximation. Though with modern computers it is easier now to use non-parametric or other tools and we do not need to depend on the normal as much (but history/inertia/etc. keeps us using normal based methods).

If you square a variable that comes from a standard normal distribution then it follows a Chi-squared distribution. If you add together variables from a Chi-squared you get another Chi-squared (degrees of freedom change), so that means that the variance (scaled) follows a Chi-squared.

It also works out that a function of the likelihood ratio follows a Chi-squared distribution asymptotically if the null is true and other assumptions hold.

A standard normal divided by the square root of a chi-squared (and some scaling parameters) follows a t-distribution, so the common t-statistic (under the null hypothesis) follows the t.

The ratio of 2 Chis-squareds (divided by degrees of freedom and other considerations) follows an F-distribution. The anova F tests are based on the ratio of 2 estimates of the same variance (under the null) and since variances follow a Chi-squared, the ratio follows an F (under the null and assumptions holding).

Smart people worked out these rules so that the rest of us can apply them. A full math/stat course will give more of the history and derivations (and possibly more of the alternatives), this was just meant as a quick overview of the more common tests and distributions.

• Thanks, this is exactly what I was looking for. I think I will put off the math-theory stats course for now though. – Stu Sep 23 '13 at 19:19

A different way to answer your question is the following sequential thinking that I would like to illustrate with a simple example:

1) Whats the null hypothesis related to the question of interest? E.g. in US, the average income is $6000 per month. 2) How can we measure the deviance from the null hypothesis based on available data? First try:$T =$Average income. The further away from 6000, the less plausible the null hypothesis is and the more we should reject it. 3) Find the distribution of$T$if the null hypothesis is true. This "null distribution" is the basis for the test decision. In our example, if the sample is large, the Central Limit Theorem tells us that$T$is approximately normally distributed with mean 6000 and standard deviation$\sigma/\sqrt{n}$, where$\sigma$is the true standard deviation of the income in US. We know$n$and$\sigma$can be estimated by the sample standard deviation$\hat \sigma$. Principally, we could now lean back and use this result to find test decisions. However, because we statisticians are nice, we usually try to modify the test statistic to keep the null distribution free of as much data dependent information as possible. In our simple example, we could use $$T' = (T-6000)/(\hat \sigma/\sqrt{n})$$ instead of$T$. This modified test statistic$T'$is always approximately standard normal if the null hypothesis is true. No matter the sample size, the hypothesised mean and the standard deviation, the test decision is always based on the same critical values (like$\pm 1.96\$). This is the famous one-sample Z-test.

There are only three reality based distributions. (1) The Binomial (2) The Multinomial (3) Abraham De Moivre's approximator to the binomial. The other distributions are 'derived' expressions with very limited dynamic range and very little contact with reality. Example. A statistician will tell you your data fits to a Poisson Distribution. He will actually believe the Poisson distribution has some kind of 'stand alone' reality. Truth is, the Poisson Distribution approximates the binomial for very small and very large amounts of skew. Now that we all have computers there is no reason to call upon approximators. But, sadly, old habits die hard.

• An interesting and thought-provoking thesis, but ultimately less than helpful in this context. Moreover, its truth would seem to rest on an idiosyncratic and limited idea of "reality-based." (To justify that allegation of limited, consider--among many examples--what it would take to derive distributions like the hypergeometric or Benford from the three distributions named here.) – whuber Sep 24 '13 at 21:51
• I don't see how a computer alleviates the need to approximate the model underlying a complex process. People aren't using Poisson regression because their data were generated from a huge number of Bernoulli trials where the success probability decreases proportional to the number of trials and they just want to save their computer the trouble. They use it because it's a simple model for testing how covariates affect the mean of a count outcome. A shrewd practitioner checks the assumptions of their models but, until computers become psychic, we'll be using models to approximate reality. – Macro Sep 24 '13 at 22:11
• In the life sciences it is important to test data sets against the binomial distribution. Doing so gives us a measure of the total number of 'sources of error' which corresponds to the number of genes influencing the process. The Poisson Distribution, among others, obscures this relationship. – user10739 Sep 27 '13 at 22:15