(Disclaimer: I am aware that some form of this question has been asked many times on this site, e.g. this, this and this - I was hoping to find a definitive answer to this from Google but I am still confused by reading these links and I don't have enough privilege to comment, so here goes this question.)
Mathematically, it is necessary to assume data is normal, for t-test to be applied. (This is because of the denominator in t-statistic, where we assume sample variance distributes like chi squared after normalization)
- The most popular answer to justify "t-test is good enough for non-normal population" is to say that for large sample sizes, the sample mean is approximately normal by Central Limit Theorem. There is no guarantee on how large the sample size is needed, but we can take it on faith.
- I'd argue that by the exact same argument, we can just use z-test instead of t-test, especially since t-distribution is close to normal when n is "big", so Slutsky's Theorem would apply.
- This must mean that if we favor the t-test over the z-test, we must believe that in a transitional range where n is not big enough, the t-test is better than the z-test in some way, even for non-normal data (e.g. the t-test is more powerful than the z-test). Why? What are the assumptions that go into such statement, or is it purely empirical?