2
$\begingroup$

Why is the t-distribution independent of the true variance of the underlying function? Intuitively I would not expect this to be true. Can anyone explain why this is true, using words rather than mathematical formulas?


Here is an example of exactly what I mean:

Consider 2 functions F and G.

F returns values which are normally distributed around mean 0 with variance 1.

G returns values which are normally distributed around mean 0 with variance 5.

Consider taking a sample of size 20 from F or G.

If we take multiple samples like this from F, the sampling distribution will be Student's t-distribution with 20-1 degrees of freedom.

If we take multiple samples like this from G, the sampling distribution will be Student's t-distribution with 20-1 degrees of freedom.

Intuitively I would expect to get a different distribution.

$\endgroup$
  • $\begingroup$ It seems like you are confusing things. What can be said is that if you have a sample from a normal distribution the t statistic will not dependent on the population variance. $\endgroup$ – Michael R. Chernick May 5 '17 at 19:06
  • $\begingroup$ @MichaelChernick can you please point out the things that are wrong in my post? $\endgroup$ – Atte Juvonen May 5 '17 at 19:47
  • 1
    $\begingroup$ Far from being any different, $F$ and $G$ are identical: you can view them as producing exactly the same quantities, but merely expressed in different units of measurement. ($F$ could measure a signed distance in Tuscan canne while $G$ could measure the same distance in bracci, for instance.) The $t$ statistic is one of many possible quantities that does not change when the units of measurement are changed. $\endgroup$ – whuber May 5 '17 at 20:01
  • $\begingroup$ The T-distribution you refer to is the distribution of the test statistic $T$.The test-statistic $T$ is independent of the population variance because the population variance is a constant not a random variable. The test-statistic $T$ is asymptotically independent of the sample variances as well because of the central limit theorem. I think the test-statistic is also independent of the sample variance when the data are normal. They will not be independent when the data are non-normal. $\endgroup$ – AdamO Mar 28 '18 at 15:50
4
$\begingroup$

The first thing to note is that the sample mean and sample variance are independent. The proof of this is a bit long to include here but can be found online pretty easily. The sample mean has distribution $$\bar{X} -\mu \sim N(0, \sigma^{2} /n)$$ and the sample variance has dist $$(n-1)S^{2} / \sigma^{2} \sim \chi^{2}_{n-1}$$ With abuse of notation $$\frac{\bar{X}-\mu}{S / \sqrt{n}}= \frac{\sigma Z / \sqrt{n}}{\sigma \sqrt{\chi^{2}_{n-1}/(n-1)}/\sqrt{n}}=\frac{Z}{\sqrt{\chi^{2}_{n-1}/(n-1)}}=t_{n-1}$$ Here the Z is a standard normal, and is independent of the Chi squared term, and the final equality comes from the definition of a t random variable.

$\endgroup$
  • $\begingroup$ Hi @AdamO, from what I understand he's not asking about a two sample t-test at all. The two distributions are brought up just to contrast the fact that the a t statistic computed from either one on the null H0: mu=0 would have the same distribution. $\endgroup$ – MHankin Mar 29 '18 at 18:00
  • $\begingroup$ You're right. Somehow I read into F and G as being two pops from which one would want to compare variances. Disregard the previous comment. $\endgroup$ – AdamO Mar 29 '18 at 19:56
1
$\begingroup$

Keep in mind that the statistic is divided by the sample standard deviation so in a way you "use" this value as a (roughly speaking) standardisation tool.

Thus, if you take 20 sample values from either f then both the nominator and the denominator will be lower (most probably) compared to if you do the same from g. This will make both statistics to follow the t-distribution.

$\endgroup$
1
$\begingroup$

By definition (at least my definition), the Student's T distribution is the distribution of the variable

$$T=\sqrt n \frac {\overline{X_n} - \mu} {\sqrt{\frac 1 {n-1}\sum(X_i-\overline{X_n})^2}}$$

Where the $X_i$ are independent $N(\mu, \sigma^2)$ variables. The question is: why is this definition actually a definition? Why does the distribution of $T$ not depend on $\mu$ and $\sigma^2$?

A simple way to prove that the distribution is well-defined is to do the following algebraic manipulations. First, rewrite the sample variance by adding and removing $\mu$:

$$\sum(X_i-\overline{X_n})^2 = \sum((X_i-\mu)-(\overline{X_n}-\mu))^2$$

And now divide both parts of the fraction by $\sigma$:

$$T=\sqrt n \frac {(\overline{X_n} - \mu)/\sigma} {\sqrt{\frac 1 {n-1}\sum(\frac{{X_i-\mu}}\sigma-\frac{\overline{X_n}-\mu}\sigma)^2}}$$

The point is, we've gotten an expression of the form:

$$T=f\left(\frac{X_1-\mu}{\sigma}, ..., \frac{X_n-\mu}{\sigma}, \frac{\overline{X_n}-\mu}\sigma\right)$$

That is, we have expressed $T$ as a function of the variables $Y_i=\frac{X_i-\mu}{\sigma}$ and of $\frac{\overline{X_n}-\mu}\sigma$, which are all normally distributed variables. In fact, since $\frac{\overline{X_n}-\mu}\sigma$ can itself be expressed in terms of the $Y_i$, we can even write, for some complicated function $f$:

$$T=f(Y_1, ..., Y_n)$$

Since the joint distribution of the vector $(Y_1, ..., Y_n)$ doesn't depend on $\mu$ or $\sigma^2$ (it is $N(0, I_n)$), neither does the distribution of $T$.

$\endgroup$
  • $\begingroup$ Wouldn't there be $n+1$ elements in your 3rd display of $T$? That $n+1$-th element is not $N(0,1)$ but $N(0, 1/n)$ approximately, and it's not immediately claimed or (to me) clear the $Y_{n+1}$would be independent of any of the previous $n$ observations. $\endgroup$ – AdamO Mar 28 '18 at 15:56
  • $\begingroup$ @AdamO I removed the statement that $Y_{n+1}$ has unit variance. I'm not sure if $Y_{n+1}$ is independent from the $Y_i$, but I don't claim it is. Since $Y_{n+1}$ can be expressed as a function of the $Y_i$ ($i<n+1$), we only need independence of the $Y_i$ ($i<n+1$). $\endgroup$ – Jack M Mar 28 '18 at 16:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.