The one-sample t-Test/z-test and inference validity

The CLT tells us that as we collect the means of different samples, the sampling distribution resembles a normal distribution and this way we can infer with a CI on the sampling distribution, the population mean. Now, this can be done via bootstrapping or manual sampling, in both cases the inference is totally valid.

Now, the one-sample t-test also allows you to infer on the population mean with a confidence interval, but how is it valid if it doesn't sample with replacement (bootstrapping) or with manual sampling; meaning; how is the t-test able to infer without incurring in the required additional sampling process that the CLT needs, stated in the previous paragraph? The same question applies to a proportion z-test on categorical data.

• Do you understand how it works if you satisfy the normality assumption and don't need to rely on an asymptotic argument?
– Dave
Aug 24 '21 at 19:00
• Set aside the central limit theorem; do you understand how sampling distributions work?
– Dave
Aug 24 '21 at 19:04
• I guess I do, where is this going exactly? Aug 24 '21 at 19:05
• So you understand $\bar X$ as a random variable?
– Dave
Aug 24 '21 at 19:05
• Yes, the sample mean would be a random variable Aug 24 '21 at 19:12

The CLT says the sampling distribution of the sample mean is asymptotically normal.

From a single sample, we can estimate the mean and the standard error. These are our best guesses for each parameter. Given that each is an unbiased estimate for the mean and the standard deviation of the sampling distribution, we apply them as if they were the truth.

So, the knowledge of the asymptotic behaviour coupled with unbiased (and hopefully high precision) estimates allows us to make the inference. Inference in the case where we have a null about the mean is even simpler. We have a null (the mean is 0) and since we know the asymptotic behaviour of the sampling distribution, we can compute p values for seeing test stats at least as large as what we've seen.

• So the t-test works because it works on the assumption that the data is already normally distributed without the need to asymptotically convert non-normal distributions to a normal with the CLT? If so, when your data is not normally distributed, you would be forced to follow the manual sampling/bootstrapping procedure as you can not satisfy the t-test assumption that the data is normal? Please correct me if I'm wrong Aug 24 '21 at 19:08
• The derivation of the t-distribution of the t-statistic does rely on the normality of the original variables it's a function of, yes. If you rely on the CLT (and at least one other theorem, since the denominator of the t is also a random variable) you don't actually get to a t-distribution, but rather to a normal distribution in the limit. On the other hand, you can - under suitable conditions - demonstrate that the t distribution and the thing whose distribution you are approximating are both heading to the same destination. ... ctd Aug 24 '21 at 23:41
• ctd... In many circumstances this approximation of an approximation works perfectly well, and at least reasonably often the difference between the t distribution and the asymptotic distribution of the t-statistic is small in large samples and the larger error is in the approximation of the statistic by either of them. Aug 24 '21 at 23:41

I think one thing that is slightly tricky with the CLT is the word “asymptotic”. I used to think that it meant infinity in sample size, causing confusion as to why you would get a normal distribution rather than a single dererministic value (leaving aside whether infinte sampling is even possible, for example it might take a long time before your results become available!) But the CLT includes this “infinte” case and also where you have merely very large samples.

Under certain conditions the CLT says that the sampling distribution of the mean is asymptotically normal, with the same mean as the underlying distribution of the variables but with a variance that is $$1/\sqrt{N}$$ times as big as the underlying standard deviation. Note that as $$N$$ tends to infinity the standard deviation shrinks and you would eventually get to zero.

Once you have enough sample size to be in the asymptote, you are in normal distribution territory for the sample mean. But you may well need to use the t-distribution if you want to estimate the mean as you will unlikely know the standard deviation.

Welcome someone like @kjetil-b-halvorsen if would want to make any comments or corrections on the use of the informal summary here.