3
$\begingroup$

So suppose I'm trying to decide whether to reject the null hypothesis that the mean of some random variable $X$ is zero based on the sample mean $\hat{\mu}$ computed from $n$ IID draws of this random variable. Then normally I would do a $t$ test $$t=\frac{\hat{\mu}}{\hat{\sigma}\;/ \sqrt{n}}$$

and check to see what the probability is that one would draw a more extreme value than $t$ from the $t$ distribution.

But why even bother scaling by $\sqrt{n}$? Why not just use Student's z-distribution? After all isn't that the true distribution for the sample mean $\hat{\mu}=\frac 1n\sum_{i=1}^nX_i$? The only reason I can see to bother is so that for large $n$ Student's t distribution approximates the standard normal distribution, which might simplify computation, is this the only reason it's done?

$\endgroup$
  • $\begingroup$ good question. you might want to check out "on the transition from student's z to student's t" eisenhart 1979 $\endgroup$ – jsk Jun 5 '14 at 4:16
  • 1
    $\begingroup$ (1) What's the variance of $X_i$? (2) "Why not just use the student's z-distribution? After all isn't that the true distribution for the sample mean?" -- what is this belief based on? $\endgroup$ – Glen_b -Reinstate Monica Jun 5 '14 at 10:41
  • $\begingroup$ As a small matter of form, Student was the pseudonym used by W.S. Gosset and so should be capitalised like any surname or family name. (Edited accordingly.) $\endgroup$ – Nick Cox Aug 14 '14 at 7:53
5
$\begingroup$

Generally, when you do a statistical test, you want the null distribution of the statistic to have a stable form, so as to allow easy computation of P-values.

This is partly a historical consideration. In principle, there's no reason you couldn't use the unscaled statistic $\hat{\mu}/\hat{\sigma}$, as opposed to $\hat{\mu}/(\frac{\hat{\sigma}}{\sqrt{n}})$, and compare to the percentage points of a distribution that became successively narrower with sample size. But this would be very cumbersome if you didn't have modern computers to calculate those percentage points for you. Even today, when we do have computers that could do this calculation, having a relatively stable null distribution makes it easier to compare results from different studies involving different sample sizes. A t-statistic of 2.5 is easier to comprehend because it means roughly the same thing whether you have a sample of 100 or 100,000. You can't say the same about the unscaled z-statistic.

$\endgroup$
  • 1
    $\begingroup$ Nice answer. You have not mentionned that using either $t$ or $z$ is strictly equivalent because it is obvious. I would add it because (oddly) it is not clear for everyone, as shown by the upvotes to the first answer. $\endgroup$ – Stéphane Laurent Aug 15 '14 at 8:02
  • $\begingroup$ This holds when $\mu$=0. Otherwise to get a central t you need to subtract $\mu$. $\endgroup$ – Michael R. Chernick Jun 1 '17 at 5:10
  • $\begingroup$ You don't get the t distribution with n-1 df if you don't divide by the square root of n. $\endgroup$ – Michael R. Chernick Jun 1 '17 at 5:23
  • $\begingroup$ In the test the value of $\mu$ used is the value under the null hypothesis. $\endgroup$ – Michael R. Chernick Jun 1 '17 at 5:24
4
$\begingroup$

UPDATE: Added material to respond to some issues that arose in the comments.

As stated by the reference the OP pointed to, $\hat \mu$ does not follow Student's z-distribution - $(\hat \mu -\mu)/s$ does. But when these distributional results hold in finite samples? Only when the $X$'s are normally distributed.

So you have to wonder: Under non-normality the usual t-statistic still converges to a standard normal by the Central Limit Theorem, because under the null it is a sum of random variables centered, and scaled by their standard deviation. Technically, it is the term $\sqrt n$ that prevents it for converging to zero.

So where does the statistic you propose, $(\hat \mu -\mu)/s$, converges to? Isn't it zero? Irrespective of the true value of $\mu$?
So what good is it for a statistic in non-normal samples if it gradually "loses" its finite sample distribution and tends to the same constant?

ADDENDUM
One can search and verify that if $Z_n$ follows a Student's- $z$ distribution with $n$ degrees of freedom, and $T_{n-1}$ follows a Student's- $t$ distribution with $n-1$ degrees of freedom, then the following relation holds between them:

$$T_{n-1} =Z_{n} \sqrt {n-1} \Rightarrow Z_{n}=\frac 1{\sqrt {n-1}}T_{n-1}$$

Assume that we have a size-$n$ sample of i.i.d. random variables $\{X_i\}$ following $N(\mu, \sigma^2)$, both parameters unknown. Setting

$$\bar X_n = \frac 1n\sum_{i=1}^nX_i,\;\;\; s^2 = \frac 1{n-1}\sum_{i=1}^n(X_i-\bar X_n)^2$$

then we know that the random variable

$$\sqrt n\frac {\bar X_n -\mu}{s} \sim_{\text{finite-sample}} T_{n-1}$$

and also that

$$T_{n-1} \sim_{\text{asympt}} N(0,1)$$

The important thing to note here is that $T_{n-1}$, has a) a finite sample distribution and b) an asymptotic distribution. This means that $T_{n-1}$ remains a non-trivial random variable even asymptotically.

Let's now turn to the $Z_n$ random variable. By previous results we have

$$Z_n = \frac 1{\sqrt {n-1}} \left({\sqrt n}\cdot\frac {\bar X_n -\mu}{s}\right) \sim_{\text{finite-sample}}\frac 1{\sqrt {n-1}}T_{n-1}$$

and this quantity follows a Student's-$z$ distribution. What is its asymptotic distribution? It is obvious that $Z_n$ converges to zero, since

$$\lim_{n\rightarrow \infty} \frac 1{\sqrt {n-1}} = 0,\;\; T_{n-1} \xrightarrow{d}N(0,1)\;\; \Rightarrow Z_n \rightarrow 0$$

What does that tell us? That while $Z_n$ has a finite-sample distribution, asymptotically it collapses to the constant $0$, i.e. it does not remain a non-trivial random variable asymptotically.

So, to respond to a comment, these two statistics are not asymptotically equivalent -the one remains a random variable with a distribution, the other collapses to a constant. Therefore the tests based on $T_{n-1}$ are asymptotically valid while the tests based on $Z_n$ are not.

Should we care? After all, our samples are always of finite size, why should we care what happens asymptotically?

Well, from the above it is obvious that while the distribution of $T_{n-1}$ "stabilizes" as the sample size increases (due to the existence of an asymptotic distribution), the distribution of $Z_n$ does not, as it gradually collapses to a constant. And we do not know how does this affects the shape of the collapsing distribution, and the related probabilities.

This means that, in order for our finite-sample test using $Z_n$ to be valid, for each $n$ we have to calculate the probabilities-critical values that interests us... Amusingly, the OP asked

"why even bother scaling by $\sqrt n$?"

Because, the answer is,

"if you don't bother scaling by $\sqrt n$,then you will have to very much bother computing the CDF of $Z_n$ for the specific $n$"

Which approach is more time-efficient?

Now, when we move from normal to non-normal samples, the centered and scaled sample mean does not follow Student's-$t$ in the first place, and we are left only with the asymptotically valid normality result which now comes from the Central Limit Theorem (and not through Student's -$t$). In this case, using $Z_n$ even for small samples sizes introduces an unknown approximation error, and renders the statistical test results questionable.

PS: Historically, W.S.Gosset derived first Student's -$z$ distribution (since he was interested in very small sample sizes), and laboriously calculated critical values for $n\leq 10$. It was through his collaboration with R.A. Fisher that they turned from it to the Student's -$t$.

$\endgroup$
  • 1
    $\begingroup$ I wonder why the downvote? Could the person who downvoted provide some suggestions for improvement at least? Or point to possible mistakes in this answer? $\endgroup$ – COOLSerdash Aug 14 '14 at 6:36
  • 1
    $\begingroup$ @COOLSerdash Thank you for asking the question I would also ask. Since I cannot find any mistake in the answer, perhaps the downvoter considered as negative that I have not provided mathematical formulas and proofs, but only verbal conversation... Apparently (s)he has no interest in communicating the reasons for the downvote, so I guess we will never know. $\endgroup$ – Alecos Papadopoulos Aug 14 '14 at 14:31
  • 1
    $\begingroup$ @AlecosPapadopoulos The tests are equivalent : one rejects the test based on $t$ if and only if one rejects the test based on $z$. Hence if one is asymptotically valid, the other one is too. $\endgroup$ – Stéphane Laurent Aug 14 '14 at 20:33
  • 1
    $\begingroup$ @AlecosPapadopoulos Sorry I was not the one who downvoted but this time I do, because you're claiming there are some differences between two equivalent tests. $\endgroup$ – Stéphane Laurent Aug 15 '14 at 6:32
  • 1
    $\begingroup$ @AlecosPapadopoulos one has the normal approximation $t_n \approx {\cal N}(0,1)$ equivalent to the normal approximation $z_n \approx {\cal N}(0,1/\sqrt{n-1})$. Try to illustrate using simulations what you are claiming. You can't, because the tests are the same. $\endgroup$ – Stéphane Laurent Aug 15 '14 at 6:38
3
$\begingroup$

If I correctly understand, you ask why one uses $$ t=\frac{\hat{\mu}}{\hat{\sigma}\;/ \sqrt{n}} $$ and not $$ z= t\;/ \sqrt{n-1}. $$ Using the test based on $t$, one rejects the null hypothesis when $|t|>t^*$ where $t^*$ is a quantile chosen in order that the test achieves the nominal level.

Using the test based on $z$, one would similarly reject the null hypothesis when $|z|>z^*$ where the quantile $z^*$ is simply given by $z^*= t^*\;/ \sqrt{n-1}$.

Thus, using either $t$ or $z$ provides exactly the same test: the conclusion drawn from the test based on $t$ is the same as the one drawn from the test based on $z$. A hypothesis test is nothing but a rejection rule, the rejection rules are the same, hence the tests are exactly the same. This is obvious, but I emphasize this point because the upvoted answer to the OP claims there are some differences !

One advantage of using $t$ is that the values of $t^*$ are more convenient than those of $z^*$. The approximation $t \sim {\cal N}(0,1)$ as $n \to \infty$ is well-known, and it is well known how $t$ "looks like" ${\cal N}(0,1)$ for $n$ growing from $1$ to $\infty$ and in particular we know that $t^*$ decreases as $n$ decreases. For high values of $n$ the values of $t^*$ are close to the quantile of ${\cal N}(0,1)$, and it is not hard to learn by heart the (rounded) values of $t^*$ for small values of $n$.

In other words, the table of the $z$-distribution, giving the $z^*$, would be less digestible than the table of $t$-distribution.

$\endgroup$
  • $\begingroup$ Can the downvoter explains its downvote ? $\endgroup$ – Stéphane Laurent Aug 15 '14 at 12:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.