# How to understand taking a simple sample to compute the Confidence Interval, using CLT?

Suppose there is a population distribution having mean M and standard deviation SD. We want to estimate the Confidence Interval by using a single sample using CLT. We take a sample and compute it's mean(say SM) and standard deviation(say SSD). The distribution of Sample Means is normally distributed with a standard deviation close to SD/sqrt(n) where n is the sample size, and mean(say MSM) close to M.

We can understand this process in two ways:

1. That what we are trying to compute is the range MSM +/- 2SD/sqrt(n). This is what our Confidence Interval is.

Now:

• We approximate(replace) MSM with SM.

• SD and SSD are quite close, so SD/sqrt(n) can be written as SSD/sqrt(n).

Now, we can compute the Confidence Interval(MSM +/- 2SD/sqrt(n)), using the sample mean and sample standard deviation.

But the problem with this understanding is that as a result of our process using CLT and a single sample, we can end up taking a sample having an exact mean of MSM + 2SD/sqrt(n) or MSM - 2SD/sqrt(n), with a 95% probability. In such cases, the Confidence Interval we will be computing are: MSM to MSM + 4SD/sqrt(n) or MSM - 4SD/sqrt(n) to MSM, which will be 50% off the confidence interval we were meant to calculate. So, how one solves this contradiction? The second way might have the answer.

2. That we assume MSM is actually quite close or same as M (Population Mean). Then we say that we have a 95% chance of taking out a single sample which has the mean within range MSM +/- 2SD/sqrt(n), in which case it will be 100% certain that our MSM(or Population Mean) is within 2SD/sqrt(n) of the SM (Sample Mean). In other words, there is a 95% chance of our Population Mean lying within 2SD/sqrt(n) of the SM. So, that is how it comes out to be our Confidence Interval.

But if the second approach is indeed correct, that would mean forgoing the idea that the distribution of Sample Means provides a Confidence Interval of the Population Mean(M) around it's mean(MSM). And will that be correct?

Thanks

So let's start by breaking down the formula for confidence intervals (CIs), Let's consider this in a little more generality then in your question. Let's say we want a confidence interval for an estimator of some parameter of the population, $$\hat\theta$$ (this could be the sample mean, or a regression coefficient, etc.). The population (true) parameter will be $$\theta_0$$. The CI is given by

$$[\hat\theta - c/\sqrt{n}, \hat\theta + c/\sqrt{n}]$$

$$c$$ is a critical value that we need to choose so that the CI will include $$\theta_0$$ with probability $$1-\alpha$$. This is our significance level. Setting $$\alpha = 0.05$$ is where you get that 95% figure. So what we want is that,

$$\Pr[\theta_0 \in [\hat\theta - c/\sqrt{n}, \hat\theta + c/\sqrt{n}]] = 1-\alpha$$ We can rewrite this probability as, $$\Pr[\sqrt{n}|\hat\theta - \theta_0|\leq c]$$

This is easy to see just subtract out $$\hat\theta$$ from the upper and lower bounds and multiply by $$\sqrt{n}$$.

Now I am not sure how much statistics that you have but $$\sqrt{n}|\hat\theta-\theta_0|$$ is a very common expression. In many estimators, this will have an asymptotic distribution that can be proven to be normal using the CLT. Given this normal asymptotic distribution, we can use what we know about normal distributions to pick a $$c$$ in which we have $$1-\alpha$$ coverage.

To see this in action let's go back to your example of normal means. Let's let $${X_1, \dots, X_n}\sim N(\mu, \sigma^2)$$ and iid. Let us assume both the mean and $$\sigma^2$$ are known for simplicity. Then $$\hat\theta=\bar X_n$$ or the sample mean. Then we have,

$$\Pr[\frac{|\bar X_n - \mu|}{\sigma \sqrt{n}}\leq c/\sigma] = \Pr[|Z|\leq \frac{c}{\sigma}]$$

To see that notice that all we did was divide by $$n$$ and $$\sigma$$ on both sides (noticing both are positive). Now we immediately know that $$\frac{|\bar X_n - \mu|}{\sigma \sqrt{n}}$$ is a z-score and is distributed normal and so we replace it with $$Z\sim N(0,1)$$. In other cases we will use the fact $$\sqrt{n}{(\hat\theta-\theta)}\to^d N(0,V)$$ to do the same thing.

Now that we are this far all we need to do is invert the test and determine what $$c$$ should be for our desired confidence value. That is we want to find $$z_\alpha$$ such that,

$$\Pr[Z\leq z_\alpha]=\alpha$$

To do this we need to look at the inverse of the normal pdf which is what you do when you look through a Z-table. Once we find this value we can set $$c/\sigma=\sigma z_{1-\alpha/2}$$ (two-sided stat). To get the confidence set,

$$[\bar X_n - \frac{\sigma}{\sqrt{n}}z_{1-\alpha/2}, \bar X_n + \frac{\sigma}{\sqrt{n}}z_{1-\alpha/2}]$$

which will have $$1-\alpha$$ coverage.

So to wrap up this is where the normal approximation is coming from and how the CLT is used. You will notice we never replaced the population mean (parameter) with the sample mean (parameter). Rather we just found a region around the sample parameter for which we can be sure (to a $$1-\alpha$$ level of confidence) that the population parameter lives in using our normal approximation.

• Where the issue is: (Xn with bar) is a parameter, not a single value. You urself say: "we just found a region around the sample parameter for which we can be sure that the population parameter lives..". We are taking a single sample, NOT a lot of samples. If u read my question properly(I have edited it a bit), my issue is precisely that. Also, you can read the other answer and my comment on it, and tell ur views about that. May 24 at 9:22
• Based on the comments in the other answer it sounds like the speaker may be generating multiple samples. This does not really change the procedure for building confidence sets. So the above should still apply. May 24 at 16:08

"M and SM are quite close and so, M can be replaced with SM." No, M is nowhere replaced by SM, and you don't need to assume they are close. The CLT makes a statement about the distribution, i.e., the variation of the SM, its distance from M. It doesn't require anywhere that this distance is small (although it is expected to become small for large n as a consequence of the theorem).

"Let us suppose that the normal distribution of sample means, has an exact mean of MSM." The mean of the distribution of the sample means is always M, even without CLT, normality, and asymptotics (as long as M exists).

"...what we are trying to compute is the range MSM +/- 2SD/sqrt(n)." No, we're fine with SM +/- 2SD/sqrt(n), no need to involve MSM.

"So, how one solves this contradiction?" By not introducing MSM in the first place.

"2. That we assume MSM is actually quite close or same as M (Population Mean)." These are the same anyway, see above.

• So, if I understand you correctly, what you are saying in essence is that the second approach is indeed correct and we should forego the idea that the distribution of Sample Means provides a Confidence Interval of the Population Mean(M) around it's mean(MSM). But if that is correct, then is what is said from 2:00 - 2:40 in this video is wrong?: youtube.com/… May 24 at 8:44
• I have edited my question. You can take a look. May 24 at 9:33
• I had a quick look at 2:00 of the video but don't have time to watch it in full, so I can't comment on it. Your "second approach" still is based on a distinction between M and MSM, so I don't think this is correct - it may be more confused than incorrect though. May 24 at 12:08
• PS: It may be that the speaker in the video simulated many samples and showed the mean of those, so his "MSM" may be the simulated one, therefore slightly different from the precise one, which, as I wrote, is equal to M. Chances are this is correct but you may have confused the simulated MSM with the theoretical one. May 24 at 12:09
• No, actually in my second approach I clearly say that MSM is same as M, so I am presuming both of them to be the same or very close to each other. And I think you have hit the nail on the head: When trying to take a single sample and deriving CI from it, we have to take the theoretical MSM which is same as M. And when taking a lot of samples and then calculating CI through their sample means - first approach will be applicable. Please correct me if I am wrong. May 26 at 0:21