It is commonly stated that if $X_i$ are iid $N(\mu, \sigma)$, then with $\hat{X}$ the sample mean, and $\hat{S}$ the sample error (sample standard deviation), then $\frac{ \hat{X}-\mu}{\hat{S}/\sqrt{n}}$ follows a t distribution with $n-1$ degrees of freedom.

e.g., wikipedia link states it, and another Wikipedia link states it as the natural way a t distribution arises, but nothing proves this fact.

How do we know/prove this?

The definition of a t distributed variable is $\frac{U}{\sqrt{W/(n-1)}}$ with $U$~$N(0,1)$, $W$~$\chi^2(n-1)$.

So we can write $$\frac{ \hat{X}-\mu}{\hat{S}/\sqrt{n}} = \frac{ \frac{\hat{X}-\mu}{\sigma/\sqrt{n}}}{ \frac{\hat{S}}{\sigma}} = \frac{U}{D}$$.

$U$ is $N(0,1)$ b/c $X_i$ are, meaning $\hat{X}\sim N(\mu, \sigma/\sqrt n)$.

We need to show that the bottom, $D = \sqrt{W/(n-1)}$ with $W = \frac{(n-1)S^2}{\sigma^2} \sim \chi^2(n-1)$. Looking at other post here, it seems the proof is not quite complete, but it gives a good start. Basically, the proof is: (1.) define auxiliary variable $V=\sum_{i=1}^n(\frac{X_i-\mu}{\sigma})^2$, clearly $\chi^2(n)$. (2.) Algebra shows that $V= W + Y$, where $W = \frac{(n-1)\hat{S}^2}{\sigma^2 }$ (quantity of interest) and $ Y = (\frac{n(\bar{X}-\mu)}{\sigma})^2$ which is $\chi^2(1)$. (3.) Conclude that this implies $W$ is $\chi^2(n-1)$.

So I can follow (1.) and (2.). I do not think (3.) follows directly from (2.) without justification. (e.g., if $W$ and $Y$ were independent, I think the relationship above shows $W$ is $\chi^2(n+1)$!

Overall, how do we prove (3.)? OR Please provide another proof of this whole thing. I just want to understand this rigorously.


1 Answer 1


For $\zeta\sim \mathcal N(0,1),~ \chi^2\sim\chi^2_{n},$ both being independent to each other, \begin{align}\textrm{Fisher's t} := ~ \frac{\zeta}{\sqrt{\frac{\chi^2}n}}\sim t_n.\tag I\end{align}

Now take $\zeta:= \frac{\bar x-\mu}{\sigma/\sqrt n}\sim \mathcal N(0, 1), \chi^2 :=\frac{\sum(x_i-\bar x) ^2}{\sigma^2}=\frac{ns^2}{\sigma^2}\sim \chi^2_{n-1},$ both being independent (to be proved)

\begin{align} \text{Student's t}&:= \frac{\sqrt n (\bar x-\mu)}{\sigma}\times \frac\sigma{\sqrt{\sum(x_i-\bar x) ^2/(n-1)}}\\&= \frac{\bar x-\mu}{S/\sqrt n} \sim t_{n-1}.\tag{II} \end{align}

Fisher's lemma: If $X_i\mapsto Y_i$ via an orthogonal linear transformation, where $X_i, ~i\in\{1, 2,\ldots, n\}$ are independent $\mathcal N(0, \sigma^2), $ then $Y_i\sim \mathcal N(0, \sigma^2)$ and independent.

The proof is easy. Let $\mathbf A$ be the matrix corresponding to the transformation i.e. $\bf Y = AX; $ since $\bf A$ is orthogonal, $\bf Y^\mathsf TY = X^\mathsf TX$ and $\bf A^\mathsf TA= I. $ Now one can evaluate the joint density of $Y_i$ using the relation stated; the jacobian would yield $1$ and the rest would follow.

Now, take $\bf A$ such that

$$a_{11} = a_{12} =\ldots= a_{1n}= \frac1{\sqrt n}; $$ this would imply $$y_1 = \sqrt n \bar x. $$

Now, as the transformation is orthogonal,

\begin{align} \sum y_i^2 &= \sum x_i^2\\ &= \sum (x_i-\bar x) ^2 +\underbrace{ n\bar x^2}_{=y_1^2}\\\implies \sum_{i= 2}^n y_i^2 &= \sum_{i=1}^n (x_i -\bar x) ^2; \end{align}


$$\sum_{i=1}^n (x_i -\mu) ^2= \sum_{i= 2}^n y_i^2 + n(\bar x - \mu)^2.$$

Therefore, following Fisher's lemma and the results found,

\begin{align}\mathrm dG(y_1, y_2, \ldots,,y_n) &= \frac1{\sigma^n(2\pi)^{n/2} }\exp\left [-\frac1{2\sigma^2}\left\{ \sum_{i= 2}^n y_i^2 + n(\bar x - \mu)^2 \right\}\right]|\mathcal J|\prod \mathrm dy_i\\ &= \left[\frac{1}{\sqrt{2\pi}(\sigma/\sqrt n) }\exp\left\{-\frac{ n}{2\sigma^2}(\bar x -\mu)^2\right\}~\mathrm d\bar x\right]\times \left[\frac{1}{\left(\sigma{\sqrt {2\pi}}\right)^{n-1} }\exp\left\{-\sum_{i=2}^n\frac{ y_i^2}{2\sigma^2}\right\}~\prod \mathrm dy_i\right]; \end{align}

proving the independence of $\bar X$ and $\sum_{i=2}^n Y_i^2 = ns^2.$

Now, since $Y_i \sim \mathcal N(0, \sigma^2), ~i\in\{2, 3,\ldots, n\}, $

$$ \sum_{i=2}^n \frac{Y_i^2}{\sigma^2} = \sum_{i=1}^n \left(\frac{X_i -\bar X}{\sigma}\right)^2\sim \chi^2_{n-1}.$$


Fundamentals of Mathematical Statistics, S. C. Gupta, V. K. Kapoor, Sultan Chand & Sons, 2014.

  • $\begingroup$ So if you reduce to the case \mu = 0, \sigma =1 initially (WLOG), then the proof required is completed as soon as you derive \sum_{i>1} Y_i^2 = sum_{i=1}^n (X_i - \barX)! very slick!. As for the independence (which doesn't seem to be needed but we get for free from Fisher's Lemma): I think it is easy to show that if random variables Y_1, ..., Y_n are all mutually independent, then for any i, j, Y_i is independent of Y_j^2 and Y_1 independent of \sum_{i>1} Y_i. This means Y_1^2 is independent of sum_{i>1} Y_i^2. $\endgroup$ Commented Aug 25, 2022 at 18:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.