Hypothesis testing: Use t-test when sample size is small or variance is unknown, but not both

The t-test is often used in hypothesis testing when the sample size is small (less than 30) because its parameterization by degrees of freedom allows the greater uncertainty to be accounted for. Many online information sources, however, including answers in Cross Validated, say t-tests and z-tests require approximate normality in the underlying population or random variable. This Wikipedia section says that for a one-sample t-test, the underlying population or random variable does not need to be normal if the sample is large enough that sample mean is normally distributed due to the Central Limit Theorem (CLT).

Would it be fair to say that the t-test can be used: (i) when sample sizes are small or (ii) when the underlying population or random variable is not normal, but not both (i) and (ii) at the same time? That is, the above "or" should be an "exclusive or"?

P.S. I did not mention that the t-test is used when we do not know the standard deviation of the underlying population or random variable. But the above still applies, i.e., the t-test is inappropriate if conditions (i) and (ii) both apply.

• As Wikipedia says, the t-test requires the distribution of the sample means to be normal. But that is the case for sure if the random variable itself is normal. If it is not, the distribution of the means is often approximately normal for larger sample sizes as the CLT states. Thus, it is an exclusive or (presuming the conditions for the CLT hold). Aug 7, 2022 at 4:47
• That's the way I interpret it. I just wasn't sure because so many examples online don't highlight this caveat. They just dive into the mechanics of hypothesis testing. Thanks for confirming Aug 7, 2022 at 16:44

1 Answer

The CLT has to do with type I assertion probability $$\alpha$$ being approximately correct (and even then the sample size must be huge for it to work if you have high asymmetry) but offers no protection against high $$\beta$$ (low power). The $$t$$-test can be used any time you have confidence that its normality/equal variance assumptions are true, but you'll be getting a lot of the power from the assumptions when $$n$$ is small. A Bayesian $$t$$-test recognizes uncertainty with regard to both normality and the variance ratio. More here. Or use a nonparametric method to test a more general hypothesis.

• Thanks. I'm going to look things up to fully appreciate this... Aug 7, 2022 at 16:47