Suppose there is a population distribution having mean M and standard deviation SD. We want to estimate the Confidence Interval by using a single sample using CLT. We take a sample and compute it's mean(say SM) and standard deviation(say SSD). The distribution of Sample Means is normally distributed with a standard deviation close to SD/sqrt(n) where n is the sample size, and mean(say MSM) close to M.
We can understand this process in two ways:
That what we are trying to compute is the range MSM +/- 2SD/sqrt(n). This is what our Confidence Interval is.
We approximate(replace) MSM with SM.
SD and SSD are quite close, so SD/sqrt(n) can be written as SSD/sqrt(n).
Now, we can compute the Confidence Interval(MSM +/- 2SD/sqrt(n)), using the sample mean and sample standard deviation.
But the problem with this understanding is that as a result of our process using CLT and a single sample, we can end up taking a sample having an exact mean of MSM + 2SD/sqrt(n) or MSM - 2SD/sqrt(n), with a 95% probability. In such cases, the Confidence Interval we will be computing are: MSM to MSM + 4SD/sqrt(n) or MSM - 4SD/sqrt(n) to MSM, which will be 50% off the confidence interval we were meant to calculate. So, how one solves this contradiction? The second way might have the answer.
That we assume MSM is actually quite close or same as M (Population Mean). Then we say that we have a 95% chance of taking out a single sample which has the mean within range MSM +/- 2SD/sqrt(n), in which case it will be 100% certain that our MSM(or Population Mean) is within 2SD/sqrt(n) of the SM (Sample Mean). In other words, there is a 95% chance of our Population Mean lying within 2SD/sqrt(n) of the SM. So, that is how it comes out to be our Confidence Interval.
But if the second approach is indeed correct, that would mean forgoing the idea that the distribution of Sample Means provides a Confidence Interval of the Population Mean(M) around it's mean(MSM). And will that be correct?