If I know that the population is normally distributed, and then take small samples from this population, is it more correct to claim that the sampling distribution is normal or instead follows the t distribution?
I understand that small samples tend to be t distributed, but does this only apply when the underlying population distribution is unknown?
Thanks!
1) a set of random observations from a population with distribution $F$ are samples from that distribution. So even single values sampled from a normal population are normally distributed. (Well, speaking slightly more strictly, the random variable that represents the single draw is the thing that's normally distributed.)
2) If the observations are independent draws from a normal distribution, the sample means are normal. (If they're dependent, it matters what the dependence structure is.)
3) Here's something that will be t-distributed, if the data are i.i.d draws from a normal population: t-statistics. (We get something other than normal because there's a numerator and a denominator)
I understand that small samples tend to be t distributed
This is a mistaken understanding. On what is this understanding based?
[This seems to be such a common misunderstanding that I can only assume it's in some popular or once-popular book somewhere. If you do find such a book, post details in your question or in a comment, because I'd love to know where it comes from.]
If you intend to take a value from a normally distributed population, that value has the same probability density function as that of the population. So any draw $x_{i}$ from a population $X \sim N(\mu, \sigma ^{2})$ will be drawn from the same population distribution $N(\mu, \sigma ^{2})$
So that means that small samples are still distributed Normal, right? Well, sure, in that if each draw is from a Normal distribution, it will itself have a Normal distribution (before we actually take the draw, at least).
It seems like you're asking about $\bar{x}$, since we're talking about samples, t-distributions, and the like. $\bar{x}$ isn't is still Normal for small samples, even though because each observation $x_i$ has a Normal distribution. Why? Because it's just a sum of other Normal random variables!
Glen_b made a nice catch where I conflated $\bar{x}$ and the $t$-statistic. It's important to note that while $\bar{x}$ is still Normal for any sample size (if the population it's sampled from is Normal), $t$ statistics constructed from a Normal sample aren't Normal for small sample sizes. Why?
Well, we have two distinct cases here. It is possible that the distribution is already known, in which case we know the true value of $\sigma^{2}$. It is also possible that $\sigma^{2}$ is not known, in which case we will have to estimate it.
1: We know $\sigma^{2}$. This means we can use a $z$ statistic calculated directly from population parameter $\sigma^2$.
If we are certain about the true value of $\sigma^{2}$, then we can perform e.g. hypothesis testing on $\bar{x}$ using a distribution $N(\mu, \frac{\sigma ^{2}}{\sqrt{n}})$. In particular, we can standardize it, transforming it into a value $Z$, for which the distribution is $N(0, 1)$ And if we know the value of $\sigma^{2}$, then we can just use the Standard Normal distribution for our calculations. It's Normal, no matter how large or small our sample might be!
2: We don't know $\sigma^{2}$, and so we estimate it by $s^2$.
If we don't know $\sigma^{2}$, then we need to substitute the calculated value of an estimator for the true population value. Typically, that will be $s^2$, the sample variance. But the sample variance has its own distribution, too! So we aren't actually certain about its value. And if our sample size is small, then the 'variance of the sample variance' is significant enough to affect the way $\bar{x}$ is distributed. So when we standardize $\bar{x}$, it's not Normally distributed anymore, even though all of the $x_i$ that went into calculating it are distributed Normal.
For more information, read about the definition of the t-distribution, and the distribution of the sample variance.
