Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

When estimating the mean of a population, two distributions seem to be useful, the normal distribution and the t-distribution.

Is it correct to say that $t=(\overline{x}-\mu)/{\rm SE}(\overline{x})$ follows a normal distribution for ANY population (not just normally distributed), as long as the samples sizes are significant in size (by means of the central limit theorem) ?

And, is it correct that t follows a t-distribution when the sample size is small, but then the population only if the population is normally distributed, because the central limit theorem does not apply ?

Thanks !

share|improve this question
1  
I think there is some confusion here. The standard error of the mean is not normally distributed regardless of the parent distribution. For one thing, it has to be non-negative :) You may have the role of the test statistics (e.g. the $t$ and $z$-statistics) and the standard errors mixed up. –  Macro Aug 21 '12 at 12:22
    
Hi, yes, things were completely mixed up, hopefully the reformulation is clearer ! –  vkubicki Aug 21 '12 at 15:14

2 Answers 2

up vote 11 down vote accepted

There's a subtle issue here that is not mentioned in the question regarding estimation of the standard deviation of $\overline{x}$'s sampling distribution.

Suppose you have a sample from a population with mean $\mu$ and variance $\sigma^2$. When $\sigma^2$ is known, $${\rm SE}(\overline{x}) = \sigma/\sqrt{n}$$ is exactly the standard deviation of the sample mean. In practice, you usually don't know $\sigma^2$, so you instead plug in the sample variance $\hat\sigma$ to use $${\rm SE}(\overline{x}) = \hat\sigma/\sqrt{n}$$ to estimate the standard deviation of $\overline{x}$. This distinction is actually important - when the variance is unknown, this additional uncertainty must be incorporated into the hypothesis test. This is why, even when the sample is normally distributed, the test statistic has a $t$-distribution (which has longer tails) instead of a normal distribution when $\sigma$ is unknown.

Is it correct to say that $t=(\overline{x}−μ)/SE(\overline{x})$ follows a normal distribution for ANY population (not just normally distributed), as long as the samples sizes are significant in size (by means of the central limit theorem)

This is almost correct. The population must have finite variance (i.e. not have tails that are "too long") for this to be the case. Even when the population does have a finite variance, the population distribution can have a large effect on how long until the CLT "kicks in". For shorter tailed distributions this convergence is faster. For long-tailed distributions it can take quite a while (e.g. see my example here).

Note that since we're talking about a "large sample" result here, this is true regardless of whether or not you know $\sigma$ since $\hat \sigma$ gets closer to the true $\sigma$ as the sample size increases.

And, is it correct that t follows a t-distribution when the sample size is small, but then the population only if the population is normally distributed, because the central limit theorem does not apply?

Again, assuming we're in the "$\sigma$ is unknown" world, $t$ only follows a $t$-distribution when the sample is normally distributed, which I think is what you're saying here. Related to what I said in the beginning, if $\sigma$ is known, then $t$ will have a (exact) normal distribution if the sample is normally distributed.

To summarize:

  • If $\sigma$ is known, and the population is normally distributed: $t$ has a normal distribution.

  • If $\sigma$ is unknown, and the population is normally distributed: $t$ has a $t$-distribution.

  • If the population is not normally distributed but meets the regularity requirements of the CLT: $t$ has an approximate normal distribution whether or not $\sigma$ is known. That is, the distribution of $t$ converges to a normal distribution as the sample size increases.

share|improve this answer
2  
What to say ? Thank you very much for your clarity :) ! –  vkubicki Aug 21 '12 at 16:09
    
You're welcome! –  Macro Aug 21 '12 at 16:10
    
The summary is potentially confusing: in the first bullet, $t=(\bar{x}-\mu)/\text{SE}(x)$ still has a t-distribution. The related but different statistic $z=(\bar{x}-\mu)/\sigma$ has a normal distribution. –  whuber Aug 22 '12 at 17:23
    
@whuber, I was using ${\rm SE}(\overline{x})$ to generically refer to the denominator of the test statistic (as the OP appears to), which is why I gave the definitions in the preamble to this answer. In one case, it involves $\sigma$ and in another it involves $\hat \sigma$ –  Macro Aug 22 '12 at 17:27
    
Fair enough: but in an emphasized summary it would be clearer to use a notation that is mindful of the difference. –  whuber Aug 22 '12 at 18:18

No the standard error of the mean would be related to the chi square distribution not the normal when sampling froma normal distribution. It is the sample mean normalized by the the samplestandard error that is approximately normal by the CLT and has a t distribution when the true mean is zero and the data are normally distributed.

share|improve this answer
    
@Procrastinator Yes to be precise the chi square is the distribution proportional to the sample variance for normally distributed observations. But that is not part of the answer. So I don't think I needed to qualify that in my response. –  Michael Chernick Aug 21 '12 at 11:45
    
Hi, I am sorry, but I got everything mixed up in my question... Which is much simpler than the estimation of the SEM. Thanks for your answer, I will look into the role of the Chi square in the estimation of the SEM ! –  vkubicki Aug 21 '12 at 15:15
    
@JohnDeas To the revised question Macro provides a rather complete answer. I the case when sigma is known you would use the actual sigma in the denominator rather than the sample estimate and the resulting quantity would be N(0, 1) when the sample comes from a normal dsitribution. –  Michael Chernick Aug 21 '12 at 16:01
1  
Hi @Michael, besides this answer being a response to an older (and defunct) version of the question, your statement that the sample mean has a $t$-distribution isn't true. I hope you'll consider modifying or deleting this response so as not to mislead future readers. –  Macro Aug 22 '12 at 11:37
1  
@Michael, yes of course I knew what you meant to write but subtle errors can sometimes be misleading for novices - thanks for correcting. –  Macro Aug 22 '12 at 12:17

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.