Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

If I know that the population is normally distributed, and then take small samples from this population, is it more correct to claim that the sampling distribution is normal or instead follows the t distribution?

I understand that small samples tend to be t distributed, but does this only apply when the underlying population distribution is unknown?


share|improve this question

closed as unclear what you're asking by StasK, Andy, Nick Stauner, Peter Flom Jun 7 '14 at 22:17

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question.If this question can be reworded to fit the rules in the help center, please edit the question.

I think (but I'm not sure that) the t-distribution tag wiki might answer this already... – Nick Stauner Jun 7 '14 at 4:43
The sampling distribution of what statistic? – Glen_b Jun 7 '14 at 5:05
stattheory -- if you'd like your question reopened (which will allow additional answers), you should edit your question to try to make it more clear, for example by addressing any issues raised in comments. – Glen_b Jun 9 '14 at 0:37

1) a set of random observations from a population with distribution $F$ are samples from that distribution. So even single values sampled from a normal population are normally distributed. (Well, speaking slightly more strictly, the random variable that represents the single draw is the thing that's normally distributed.)

2) If the observations are independent draws from a normal distribution, the sample means are normal. (If they're dependent, it matters what the dependence structure is.)

3) Here's something that will be t-distributed, if the data are i.i.d draws from a normal population: t-statistics. (We get something other than normal because there's a numerator and a denominator)

I understand that small samples tend to be t distributed

This is a mistaken understanding. On what is this understanding based?

[This seems to be such a common misunderstanding that I can only assume it's in some popular or once-popular book somewhere. If you do find such a book, post details in your question or in a comment, because I'd love to know where it comes from.]

share|improve this answer

If you intend to take a value from a normally distributed population, that value has the same probability density function as that of the population. So any draw $x_{i}$ from a population $X \sim N(\mu, \sigma ^{2})$ will be drawn from the same population distribution $N(\mu, \sigma ^{2})$

So that means that small samples are still distributed Normal, right? Well, sure, in that if each draw is from a Normal distribution, it will itself have a Normal distribution (before we actually take the draw, at least).

It seems like you're asking about $\bar{x}$, since we're talking about samples, t-distributions, and the like. $\bar{x}$ isn't is still Normal for small samples, even though because each observation $x_i$ has a Normal distribution. Why? Because it's just a sum of other Normal random variables!

Glen_b made a nice catch where I conflated $\bar{x}$ and the $t$-statistic. It's important to note that while $\bar{x}$ is still Normal for any sample size (if the population it's sampled from is Normal), $t$ statistics constructed from a Normal sample aren't Normal for small sample sizes. Why?

Well, we have two distinct cases here. It is possible that the distribution is already known, in which case we know the true value of $\sigma^{2}$. It is also possible that $\sigma^{2}$ is not known, in which case we will have to estimate it.

1: We know $\sigma^{2}$. This means we can use a $z$ statistic calculated directly from population parameter $\sigma^2$.

If we are certain about the true value of $\sigma^{2}$, then we can perform e.g. hypothesis testing on $\bar{x}$ using a distribution $N(\mu, \frac{\sigma ^{2}}{\sqrt{n}})$. In particular, we can standardize it, transforming it into a value $Z$, for which the distribution is $N(0, 1)$ And if we know the value of $\sigma^{2}$, then we can just use the Standard Normal distribution for our calculations. It's Normal, no matter how large or small our sample might be!

2: We don't know $\sigma^{2}$, and so we estimate it by $s^2$.

If we don't know $\sigma^{2}$, then we need to substitute the calculated value of an estimator for the true population value. Typically, that will be $s^2$, the sample variance. But the sample variance has its own distribution, too! So we aren't actually certain about its value. And if our sample size is small, then the 'variance of the sample variance' is significant enough to affect the way $\bar{x}$ is distributed. So when we standardize $\bar{x}$, it's not Normally distributed anymore, even though all of the $x_i$ that went into calculating it are distributed Normal.

For more information, read about the definition of the t-distribution, and the distribution of the sample variance.

share|improve this answer
It is real good answer that explains many more things about the small versus large samples. – subhash c. davar Jun 7 '14 at 9:16
Matt, if the data are independent normal, $\bar x$ is (demonstrably) normal, right down to $n=1$ and $n=2$, whether or not the variance is known to us. Is there some basis for your assertion otherwise? – Glen_b Jun 7 '14 at 14:38
Indeed, there are several proofs that the distribution of the sum of two independent normal r.v.s is normal here; that the average must also be normal is then straightforward. – Glen_b Jun 7 '14 at 15:35
Oops! I made a mistake, conflating $\bar{x}$ and the t statistic. Nice catch--you're very right. – Matt Jun 7 '14 at 15:38
I think I've fixed it up. $t \neq \bar{x}$, hm? – Matt Jun 7 '14 at 15:49

Not the answer you're looking for? Browse other questions tagged or ask your own question.