9
$\begingroup$

When estimating the mean of a population, two distributions seem to be useful, the normal distribution and the t-distribution.

Is it correct to say that $t=(\overline{x}-\mu)/{\rm SE}(\overline{x})$ follows a normal distribution for ANY population (not just normally distributed), as long as the samples sizes are significant in size (by means of the central limit theorem) ?

And, is it correct that t follows a t-distribution when the sample size is small, but then the population only if the population is normally distributed, because the central limit theorem does not apply ?

Thanks !

$\endgroup$
2
  • 1
    $\begingroup$ I think there is some confusion here. The standard error of the mean is not normally distributed regardless of the parent distribution. For one thing, it has to be non-negative :) You may have the role of the test statistics (e.g. the $t$ and $z$-statistics) and the standard errors mixed up. $\endgroup$
    – Macro
    Aug 21, 2012 at 12:22
  • $\begingroup$ Hi, yes, things were completely mixed up, hopefully the reformulation is clearer ! $\endgroup$
    – vkubicki
    Aug 21, 2012 at 15:14

2 Answers 2

14
$\begingroup$

There's a subtle issue here that is not mentioned in the question regarding estimation of the standard deviation of $\overline{x}$'s sampling distribution.

Suppose you have a sample from a population with mean $\mu$ and variance $\sigma^2$. When $\sigma^2$ is known, $${\rm SE}(\overline{x}) = \sigma/\sqrt{n}$$ is exactly the standard deviation of the sample mean. In practice, you usually don't know $\sigma^2$, so you instead plug in the sample variance $\hat\sigma$ to use $${\rm SE}(\overline{x}) = \hat\sigma/\sqrt{n}$$ to estimate the standard deviation of $\overline{x}$. This distinction is actually important - when the variance is unknown, this additional uncertainty must be incorporated into the hypothesis test. This is why, even when the sample is normally distributed, the test statistic has a $t$-distribution (which has longer tails) instead of a normal distribution when $\sigma$ is unknown.

Is it correct to say that $t=(\overline{x}−μ)/SE(\overline{x})$ follows a normal distribution for ANY population (not just normally distributed), as long as the samples sizes are significant in size (by means of the central limit theorem)

This is almost correct. The population must have finite variance (i.e. not have tails that are "too long") for this to be the case. Even when the population does have a finite variance, the population distribution can have a large effect on how long until the CLT "kicks in". For shorter tailed distributions this convergence is faster. For long-tailed distributions it can take quite a while (e.g. see my example here).

Note that since we're talking about a "large sample" result here, this is true regardless of whether or not you know $\sigma$ since $\hat \sigma$ gets closer to the true $\sigma$ as the sample size increases.

And, is it correct that t follows a t-distribution when the sample size is small, but then the population only if the population is normally distributed, because the central limit theorem does not apply?

Again, assuming we're in the "$\sigma$ is unknown" world, $t$ only follows a $t$-distribution when the sample is normally distributed, which I think is what you're saying here. Related to what I said in the beginning, if $\sigma$ is known, then $t$ will have a (exact) normal distribution if the sample is normally distributed.

To summarize:

  • If $\sigma$ is known, and the population is normally distributed: $t$ has a normal distribution.

  • If $\sigma$ is unknown, and the population is normally distributed: $t$ has a $t$-distribution.

  • If the population is not normally distributed but meets the regularity requirements of the CLT: $t$ has an approximate normal distribution whether or not $\sigma$ is known. That is, the distribution of $t$ converges to a normal distribution as the sample size increases.

$\endgroup$
4
  • 2
    $\begingroup$ What to say ? Thank you very much for your clarity :) ! $\endgroup$
    – vkubicki
    Aug 21, 2012 at 16:09
  • $\begingroup$ The summary is potentially confusing: in the first bullet, $t=(\bar{x}-\mu)/\text{SE}(x)$ still has a t-distribution. The related but different statistic $z=(\bar{x}-\mu)/\sigma$ has a normal distribution. $\endgroup$
    – whuber
    Aug 22, 2012 at 17:23
  • $\begingroup$ @whuber, I was using ${\rm SE}(\overline{x})$ to generically refer to the denominator of the test statistic (as the OP appears to), which is why I gave the definitions in the preamble to this answer. In one case, it involves $\sigma$ and in another it involves $\hat \sigma$ $\endgroup$
    – Macro
    Aug 22, 2012 at 17:27
  • $\begingroup$ Fair enough: but in an emphasized summary it would be clearer to use a notation that is mindful of the difference. $\endgroup$
    – whuber
    Aug 22, 2012 at 18:18
0
$\begingroup$

No the standard error of the mean would be related to the chi square distribution not the normal when sampling froma normal distribution. It is the sample mean normalized by the the samplestandard error that is approximately normal by the CLT and has a t distribution when the true mean is zero and the data are normally distributed.

$\endgroup$
6
  • $\begingroup$ @Procrastinator Yes to be precise the chi square is the distribution proportional to the sample variance for normally distributed observations. But that is not part of the answer. So I don't think I needed to qualify that in my response. $\endgroup$ Aug 21, 2012 at 11:45
  • $\begingroup$ Hi, I am sorry, but I got everything mixed up in my question... Which is much simpler than the estimation of the SEM. Thanks for your answer, I will look into the role of the Chi square in the estimation of the SEM ! $\endgroup$
    – vkubicki
    Aug 21, 2012 at 15:15
  • $\begingroup$ @JohnDeas To the revised question Macro provides a rather complete answer. I the case when sigma is known you would use the actual sigma in the denominator rather than the sample estimate and the resulting quantity would be N(0, 1) when the sample comes from a normal dsitribution. $\endgroup$ Aug 21, 2012 at 16:01
  • 1
    $\begingroup$ Hi @Michael, besides this answer being a response to an older (and defunct) version of the question, your statement that the sample mean has a $t$-distribution isn't true. I hope you'll consider modifying or deleting this response so as not to mislead future readers. $\endgroup$
    – Macro
    Aug 22, 2012 at 11:37
  • 1
    $\begingroup$ @Michael, yes of course I knew what you meant to write but subtle errors can sometimes be misleading for novices - thanks for correcting. $\endgroup$
    – Macro
    Aug 22, 2012 at 12:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.