Background: I'm giving a presentation to colleagues at work on hypothesis testing, and understand most of it fine but there's one aspect that I'm tying myself up in knots trying to understand as well as explain it to others.
This is what I think I know (please correct if wrong!)
- Statistics that would be normal if variance was known, follow a $t$-distribution if the variance is unknown
- CLT (Central Limit Theorem): The sampling distribution of the sample mean is approximately normal for sufficiently large $n$ (could be $30$, could be up to $300$ for highly skewed distributions)
- The $t$-distribution can be considered Normal for degrees of freedom $> 30$
You use the $z$-test if:
- Population normal and variance known (for any sample size)
- Population normal, variance unknown and $n>30$ (due to CLT)
- Population binomial, $np>10$, $nq>10$
You use the $t$-test if:
- Population normal, variance unknown and $n<30$
- No knowledge about population or variance and $n<30$, but sample data looks normal / passes tests etc so population can be assumed normal
So I'm left with:
- For samples $>30$ and $<\approx 300$(?), no knowledge about population and variance known / unknown.
So my questions are:
At what sample size can you assume (where no knowledge about population distribution or variance) that the sampling distribution of the mean is normal (i.e. CLT has kicked in) when the sampling distribution looks non-normal? I know that some distributions need $n>300$, but some resources seem to say use the $z$-test whenever $n>30$...
For the cases I'm unsure about, I presume I look at the data for normality. Now, if the sample data does looks normal do I use the $z$-test (since assume population normal, and since $n>30$)?
What about where the sample data for cases I'm uncertain about don't look normal? Are there any circumstances where you'd still use a $t$-test or $z$-test or do you always look to transform / use non-parametric tests? I know that, due to CLT, at some value of $n$ the sampling distribution of the mean will approximate to normal but the sample data won't tell me what that value of $n$ is; the sample data could be non-normal whilst the sample mean follows a normal / $t$. Are there cases where you'd be transforming / using a non-parametric test when in fact the sampling distribution of the mean was normal / $t$ but you couldn't tell?