Intuitive explanation for the fat tails of the t-distribution

Question

Given some standard assumptions, the test statistic

$$ \frac{\Delta\bar{X}}{\sigma/\sqrt{N}} $$

is normally distributed if $\sigma$ is known and t-distributed if $\sigma$ has to be estimated from the sample. The difference of the distributions is most pronounced for small sample sizes and in the tails (which are fatter for the t-distribution).

I wonder whether there's an intuitive explanation as to why the tails of the t-distribution are fatter when $\sigma$ is not (exactly) known.

I found two intuitive explanations for the fat tails of the t-distrbution on CV.

The first one:

The overall answer is that the T-Distribution gives a higher probability to extreme events, given a small sample size. The reasoning behind this is intuitive in that, as you can imagine, if you have relatively small sample from a population, there is a higher probability that an extreme event from the population did not "make it" into your sample. On the other hand, if you have a relatively large sample from the same population, the probability that an extreme event did not "make it" into your sample is lower, since your sample is capturing more of that population.

This is the second one:

With small sample sizes, it's relatively likely that one of the two samples will receive a slightly extreme datapoint that will skew its mean and make you think that there is a real difference between the two samples when instead nothing is going on. To correct for such small-size effect you allow more extreme values of the difference between samples to be relatively likely before claiming "significance of difference". I.e. the t-distribution, against which you map the value of the t-statistics to obtain the p-value, has fatter tails than the normal distribution. As the sample sizes increases, this correction becomes negligible and you don't need to worry about using Normal or T-distribution (or T-test vs Z-test).

Now, it seems to me that the two explanations are contradictory (though not 100% sure) and to me the second explanation is very intuitive.

In any case, both explanations would be compatible with a scenario of known population variance as well. They do not even talk about known/unknown population variance, even though if it is known we would (theoretically) obtain a normal distribution even for small samples.

So my question: is there an intuitive explanation as to why the tails of the t are (need to be) fatter precisely when $\sigma$ is unknown? In particular, why does the argument given in the quoted second explanation only apply to a case of unknown variance?

Answer 1

The heavy tails arise because in small samples, standard errors near zero have appreciable chances of arising and they are not associated with concomitantly small sample means that would cancel them out in the Student t statistic.

To appreciate this, consider small samples from a uniform distribution (which produces no extreme values of any kind). Here, from a simulation of samples of size two, are plots of the absolute value of the mean $|m|$ and standard error $se:$

The points in the scatterplot are colored according to the inverse slopes, $t = m/se,$ which are the Student t statistics. The contour lines of constant slope are, of course, lines through the origin. Near the base the inverse slopes are nearly infinite. Notice the pattern:

Large standard errors are associated with "typical" sample means (near $1/2$), where (accordingly) $t$ is not large. Conversely, the only way for $t$ to grow large is along the base of the triangle away from the origin where (a) the standard error is tiny but (b) the mean is not tiny.

This picture is not peculiar to the uniform distribution: it's just plainer than the Normal case, shown next.

(The horizontal line in the scatterplot is the expected standard error $\sigma/\sqrt{n}$ you would use for a Z statistic.)

As is well known, the standard error is uncorrelated with the mean. Nevertheless, it is still the case that

The only way for $t$ to grow large (with any appreciable probability) is along the base of the triangle away from the origin at its center where (a) the standard error is tiny but (b) the mean is not tiny.

That is, the heavy tails of the Student t distribution arise from the chance that a $(m, se)$ pair will be situated near the bottom left or bottom right of the scatterplot. In those cases the division by a near-zero standard error yields a huge value of the Student t statistic.

---

In retrospect, this is obvious in samples of size 2, because there's a $1/2\times 1/2=1/4$ chance that both values in a sample will have the same sign. Because they have the same sign, likely they are closer together than your average sample: there are your principal contributions to the heavy tails. The effect holds in large samples but quickly fades away once it's unlikely for the entire sample to have a small range.