Why is t-test more appropriate than z-test for non-normal data? (Disclaimer: I am aware that some form of this question has been asked many times on this site, e.g. this, this and this - I was hoping to find a definitive answer to this from Google but I am still confused by reading these links and I don't have enough privilege to comment, so here goes this question.)
Mathematically, it is necessary to assume data is normal, for t-test to be applied. (This is because of the denominator in t-statistic, where we assume sample variance distributes like chi squared after normalization)

*

*The most popular answer to justify "t-test is good enough for non-normal population" is to say that for large sample sizes, the sample mean is approximately normal by Central Limit Theorem. There is no guarantee on how large the sample size is needed, but we can take it on faith.

*I'd argue that by the exact same argument, we can just use z-test instead of t-test, especially since t-distribution is close to normal when n is "big", so Slutsky's Theorem would apply.

*This must mean that if we favor the t-test over the z-test, we must believe that in a transitional range where n is not big enough, the t-test is better than the z-test in some way, even for non-normal data (e.g. the t-test is more powerful than the z-test). Why? What are the assumptions that go into such statement, or is it purely empirical?
 A: Technically your argument is correct. The t- and z-test are asymptotically equivalent. However, the t-test takes into account the variation in the estimation of the standard error. Obviously, if data are non-normal, it cannot be shown that the distribution of the test statistic is a t-distribution. However, the t-distribution has heavier tails than the normal, so the t-test will be more conservative than the z-test, and the z-test may for finite samples well be anti-conservative (due to treating the standard deviation as fixed and known , which it isn't), so the t-test will be safer regarding type I error probabilities.
I believe that when simulating from specific distributions one may find occasionally that the z-test does better (i.e., better power while respecting the level), however I believe that the t-test will be better (respecting the level while the z-test is anti-conservative, or being closer to the level if both are anti-conservative) far more often. If I remember correctly I have once simulated one particular case with the t-test winning, however I didn't do anything comprehensive. I'm pretty sure you won't find a general theorem as the t-test is not universally better.
To add some intuition to this, I believe that it is generally correct that estimation of the standard error should lead to heavier tails of the distribution of the test statistic, so that in most cases the normal distribution will underestimate variation and the heavier tails of the t-distribution are more appropriate. However there may be distributions for which the finite sample distribution of the mean is subgaussian, i.e., has lighter tails than the normal (even though converging to the normal by means of the CLT). Estimation of the standard deviation in such cases will still make the tails heavier than treating it as fixed, but the normal approximation may still keep the test level in case it was overconservative enough with fixed sd. In such a (probably exceptional) case the z-test will yield better power, so it may be better. With the t-test however we are always on the safer side regarding type I error.
Note that all this assumes that (at least) second moments exist. If they don't (or if in practice tails are very heavy or there are extreme outliers), variance estimation in particular is a mess (also mean estimation but not quite as much), and both tests won't work well.
A: Now this isn't going to be a very rigorous answer, but take it for what it is.
I want to distinguish between two things. There is a difference between your data, and the statistics that you compute from your data.
T-tests are for statistics, not the data itself. The statistics will be normally distributed. Plenty of proofs of the CLT that we could refer to. It is this fact that you are leaning on that makes this work. But wait, why are we using t-tests if I said that your statistic is guaranteed to be normally distributed.
The answer is that you are using your statistic to compute a new statistic about itself. This new secondary statistic is the variance of your guaranteed normal distribution. It is an estimate not of the variance of the data, but of the statistic. This is known as a standard error.
Now, you have two sources of error about the distribution for your statistic.

*

*The original sampling error from the population when you compute your statistic

*The additional error from estimating the variance of your statistic utilizing your statistic.

You use a t-test to adjust for the uncertainty coming from source 2 of your error. The bigger your samples are, the less variance you expect will come from source 2, and your sample will approach a z-test, which would only consider errors coming from source 1, which you can't get rid of.
Essentially, doing a t-test is an act of humility where you are admitting that errors may be creeping into your confidence intervals because of the way that you are computing them. There is no way to get rid of these potential errors, so we are going to make them slightly larger than you would see with a z-test. Of course, there are derivations for why the t-distribution is the correct distribution to use to do this and so forth (I won't subject myself and you to the same tortures I did in school with gamma functions but you can look them up if you are brave enough), but that is the intuitive "why".
