What is the rationale behind using the t-distribution?

Question

(Probably a very naive question.) According to this tutorial,

Student's t-test deals with the problems associated with inference based on "small" samples: the calculated mean and standard deviation may by chance deviate from the "real" mean and standard deviation (i.e., what you'd measure if you had many more data items: a "large" sample).

So then why is the t-distribution and the t-test ideal for such a purpose?

Put differently, the t-test estimates the probability of the null hypothesis that two normal distributions have the same mean. So if we know that two sampled distributions are (reasonably) normal then why not just compute their means and compare?

Answer 1

Martijin comment is essential. Before understanding the t-test, you should understand the z-test: why and how and to adjust the scale for the difference.

Then, what motivates the t-test (from the z-test) arises from the uncertainty in estimating the standard deviation. For the intuition this text is very good about t-test and z-test.

Answer 2

An experiment, a sample from a population, will be subject to variation. However, this variation is not simple and it may be due to 1) uncontrolled and/or 2) controlled variables.

In many practical cases we wish to find out:

• not only the presence of a difference, if there is a certain target size exceeded in some measured value (e.g. a difference between two groups, or a variation of a single group from a theoretic value)
• but also the quality of the difference, whether we can ascribe the difference in the measured value to the controlled variables.

The t-score, combines those two, and is an indicator of the size of some measured value in relation to the random (uncontrolled) variation in this value.

To get an idea of the random (uncontrolled) variation, the trick is to have some measurements in which the controlled variables do not change.

---

I hope that provides some intuition. The more deeper question 'why is the t-test ideal' is more difficult.

The t-test is ideal, given that assumptions are true. And in practice it works very well for a wider range of situations. However, at what point to use some other test, and what test, such questions are hard.

From another perspective of ideal: The t-test is 'just' a hypothesis test and may not always be considered a very good practice (p-value hacking, neglecting the true research questions which are replaced by tests, etcetera). Certainly, it does not provide the entire picture and downgrades the multidimensional data to a single one-dimensional value and a final yes/no question.

Answer 3

You use a t-test, as opposed to a z-test, when the population SD is unknown so you use the sample SD instead. The larger the sample, the more tenable this is, and why the t distribution approaches the normal distribution as the sample size increases.

Answer 4

I'm guessing from your question wording that you believe the t distribution is chosen as if from a set of tools for the purposes of describing small samples. However, in this case, it's derived. That is, under the right assumptions, the t distribution arises mathematically, so there aren't any other choices. It's the unique solution to the problem of describing sample statistics under those assumptions.

If you want to see the derivation, look at the Wikipedia article about halfway down under "Characterization > Deriviation".

With that in mind, the question of whether or not to use it comes down to whether you think the assumptions are true for a particular case. Also, since the t converges to a normal, with "big enough" sample sizes, some people do exactly as you suggested. Again, it hinges on the interpretation and conclusion the statistician makes about the assumptions. (How small is "small", how "normal" do you believe your data are, etc.)

Answer 5

Since the title of your question is regarding the "Rationale behind using the T-Distribution", I will (attempt) to answer in broad terms and only go into specifics if necessary.

The overall answer is that the T-Distribution gives a higher probability to extreme events, given a small sample size. The reasoning behind this is intuitive in that, as you can imagine, if you have relatively small sample from a population, there is a higher probability that an extreme event from the population did not "make it" into your sample. On the other hand, if you have a relatively large sample from the same population, the probability that an extreme event did not "make it" into your sample is lower, since your sample is capturing more of that population.

Visually, the T-distribution looks like a Normal Distribution with a slightly lower "peak" and slightly "fatter" tails. In this picture, the Normal Distribution is the blue density and the T-Distribution is the red.

The term "small" is relative, although generally speaking is relates to samples where $N\leq30$. However, the T-Distribution is defined in terms of Degrees of Freedom, where $\textrm{degrees of freedom} = N-1$. This is where the intuition of the T-Distribution approaching the Normal Distribution as $N$ increases comes from.

Since $N$ is our denominator in calculating the mean, variance, etc, replacing the $N$ with $N~-1$ will have a larger effect when $N$ is small. For example:

1. Given a sample where $N~=~30$, the difference between the mean using $\frac{\sum_{i=1}^{30} x_i}{30}$ and the mean using $\frac{\sum_{i=1}^{29} x_i}{29}$ will be a lot greater than,
2. Given a sample where $N~=~10,000$, the difference between the mean using $\frac{\sum_{i=1}^{10,000} x_i}{10,000}$ and the mean using $\frac{\sum_{i=1}^{9,999} x_i}{9,999}$.

So, pulling back from the math to the visual representation, as $N$ increases, the T-Distribution will asymptotically approach the Normal Distrubtion, since the negative one (-1) in the definition of degrees of freedom has a ever decreasing effect as $N$ increases.

Using statistical terms, the T-Distribution allows your estimate to be unbiased when using a small sample, meaning the estimated statistic from your sample will be closer to the population statistic you are trying to estimate.

Answer 6

First of all, we must assume the data are normally distributed. Then for a sample of size $n$, the distribution of the sample average is student $t$ with $n-1$ degrees of freedom. In small samples, this corrects the tail behavior due to the uncertainty about the nuisance parameter, the standard deviation.