8
$\begingroup$

I try to understand the idea behind the t-distribution. Here are the steps that I have understood so far:

  1. We use a sample of N elements to estimate the population mean. In more details, we use the sample mean as an estimate of the population mean.
  2. We want to know how close is our estimation to the real value. Or, more specifically we want to know how large should be the interval around the sample mean so that we can say that the population mean is within this interval with a certain probability.
  3. To answer this question we assume that the values in the population are distributed according to a normal distribution with a known mean and standard deviation.
  4. Having the parameters of the distribution of the values in the population we can calculate the distribution of the sample mean as a function of the population distribution and the size of the sample.
  5. We can show that the distribution of the sample mean is also a normal distribution with the same mean as the population distribution and standard deviation given by the following formula $s = \sigma/\sqrt{N}$, where $N$ is the size of the sample.
  6. Having the distribution of the sample mean we can easily calculate the probability that the sample mean is separated from the real mean by X. Or, in other words, we can calculate the probability that the population mean is within a given interval around the sample mean.
  7. It is almost what we need. The only problem is that in real-life settings we often do not know the standard deviation of the population distribution (and this is the parameter that determines how our sample mean is distributed around the population mean).
  8. What we can do, is to replace the population standard deviation by the sample standard deviation. In other words we replace the exact and unknown parameter by our approximate estimate of it.

So, this is where I am so far. By replacing the population STD by sample STD, we make our estimate of the distribution of the sample mean worse. And to "compensate" this "wrong" value of the distribution's parameters, we change the distribution shape (we say it is not normal distribution any more, it is a t-distribution). But what exactly is distributed according to the t-distribution? When we know the population STD, we know how the sample mean is distributed around the population mean. Now we do not know the population STD, but it does not change the distribution of the sample mean around the population mean!

$\endgroup$
7
$\begingroup$

You are very close...

If $X_1, \dots, X_n$ is a sample of i.i.d normal observations with mean $\mu$ and variance $\sigma^2$, then the standardized mean $$ \frac{\bar X_n-\mu}{\sigma/\sqrt{n}} $$ is standard normal. Now, as you pointed out, in reality we never know $\sigma$. So we replace $\sigma$ by its sample estimate $S$ and consider the "studentized" mean $$ T = \frac{\bar X_n-\mu}{S/\sqrt{n}} $$ instead. This random variable is slightly different from the one above. Consequently, its distribution is slightly non-normal, namely Student with $n-1$ degrees of freedom.

For not too small $n$, $S$ is close to $\sigma$ (that's the consistency of the sample standard deviation). Then, the standardized mean is very close to the studentized one. This explains why the Student distribution with many degrees of freedom looks like the normal.

The studentized mean is the starting point to derive confidence intervals and hypothesis tests for $\mu$.

Example: To find a lower 95% confidence limit $\bar X_n -c$ for $\mu$, you solve the following equation $$ P(\bar X_n -c \le \mu) = 0.95 $$ for $c$. To do so, you try to modify the equation in the probability so that the studentized mean appears (try to figure out the substeps): $$ P(T \le \frac{c}{S/\sqrt{n}}) = 0.95. $$ Then you use the fact that $T$ has a Student distribution with $n-1$ df to get rid of the probability: $$ \frac{c}{S/\sqrt{n}} = qt_{0.95;n-1}, $$ where $qt_{0.95;n-1}$ is the corresponding 95% quantile. Thus, $$ c = \frac{S}{\sqrt{n}} \cdot qt_{0.95;n-1} $$ and the (famous) lower confidence limit follows: $$ \bar X_n - \frac{S}{\sqrt{n}} \cdot qt_{0.95;n-1} $$

$\endgroup$
  • $\begingroup$ It means that instead of speaking about sample mean (X_n), we speak about "standardized mean". We can say that the distribution of the standardized mean is normal one with mean zero and STD equal to 1. No we defined another variable by replacing population STD by sample STD and say that this new variable is distributed according to the t-distribution. OK. The last thing that I do not understand is why we do not replace population mean by sample mean. If we do not know sigma we probably also do not know the mu. $\endgroup$ – Roman Jun 18 '15 at 8:51
  • $\begingroup$ We do! But most interesting questions about $\mu$ like "in what range will $\mu$ be with high certainty" (-> confidence interval) or "is $\mu$ really different from 0" (-> hypothesis test) are answered by using the fact that the studentized mean follows a Student distribution. You cannot answer questions like that only by looking at the estimate. $\endgroup$ – Michael M Jun 18 '15 at 9:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.