2
$\begingroup$

The central limit theorem states that the means of many different samples with n>30 from a population that follows an arbitrary distribution are normally distributed.

If you now have the information from only one sample, how do you calculate the confidence interval, which states that with 95% the true mean value of the population is in this interval? The normal distribution of the mean values is not available, since only one sample was taken, with only one mean value $\overline{x}$ and standard deviation s.

The estimated value for the mean is simply $\overline{x}$, the estimated value for the standard deviation is simply s, that is trivial. But we now want to have an interval with a confidence of, say, 95%. Let's imagine the normal distribution that results from many samples. We don't have the results of many samples, but let's imagine this normal distribution. In the middle is $\mu_\overline{x}$ and to the left and right of it are the standard deviations, which of course we don't know either. Now we think of a range as being drawn in, which contains 95% of all sample means. Exactly this range we can also form around our determined sample mean $\overline{x}$ so that we can state: with 95% certainty the mean of all sample means (i.e. the population mean) lies in this area.

So what we need to calculate now is the 95% interval of the normal distribution, because we can then "put" this range over the sample mean x to get the limits of the interval. This is:

Lower limit = $\overline{x}$ - 1.96 * $\sigma_\overline{x}$

Upper limit = $\overline{x}$ + 1.96 * $\sigma_\overline{x}$

Whereas $\sigma_\overline{x}$ denotes the standard deviation of the normal distribution. How do you get the $\sigma_\overline{x}$? I have read that it can be calculated by dividing the standard deviation s of the population by $\sqrt{n}$ . But since we don't have the sigma of the population, we divide the standard deviation s (of the sample) by $\sqrt{n}$. But why can we just use s here? Why can we use just the standard deviation of the sample to calculate the interval length of the normal distribution? A too high standard deviation of the sample makes the interval too large, a too low standard deviation makes the interval too small.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ It's our best guess ("estimate" in statistical language) of the population value. $\endgroup$
    – Dave
    Aug 26, 2021 at 16:20
  • $\begingroup$ if the observations in your sample vary a lot, we are less certain where the true mean lies. $\endgroup$
    – panda
    Aug 26, 2021 at 17:33
  • 3
    $\begingroup$ "The central limit theorem states that the means of many different samples with n>30 from a population that follows an arbitrary distribution are normally distributed." ... Well no, most of those things are not what the actual CLT says. You don't need many samples, the statement of the theorem has nothing to do with n>30, but instead with asymptotic behaviour of a standardized mean or sum, and the distribution cannot be completely arbitrary (there are some conditions). Just about any undergrad level book on mathematical statistics should cover a basic form of the theorem. $\endgroup$
    – Glen_b
    Aug 27, 2021 at 4:59

2 Answers 2

Reset to default
3
$\begingroup$

I think the sequence of your story is incorrect (the high level picture). From a high level perspective, you start first with a (random) sample from which you would like to draw inferences about a given population.

Still from a high level perspective, the CLT states that, given a sufficiently large sample size, the sampling distribution of the mean for a variable will approximate a normal distribution. It states also that as the sample size increases, the mean of the sampling distribution equals the population mean and the standard deviation equals $\frac{\sigma}{\sqrt{n}}$ Where $\sigma$ is the population standard deviation and n the sample size (of the sampling distribution).

You can find an explanation for $\frac{\sigma}{\sqrt{n}}$ here : https://stats.stackexchange.com/a/60486/321901

Most of the time, we do not know the population standard deviation $\sigma$ so we estimate it through the standard error (of the sampling distribution) $se = \frac{s}{\sqrt{n}}$ where s is the standard deviation of the sample (then t-test must be used instead of z-test... but it deserves more than a "high level" answer).

Improve this answer
$\endgroup$
0
$\begingroup$

We could apply S there since this estimate of sigma (std err of population) is unbiased and consistent estimate.

Improve this answer
$\endgroup$
5
  • 2
    $\begingroup$ $s$ is not an unbiased estimate of $\sigma$. $s^2$ is unbiased for $\sigma^2$, and then Jensen's inequality says that $s$ is biased for $\sigma$. $\endgroup$
    – Dave
    Aug 26, 2021 at 16:25
  • $\begingroup$ Yes, we use instead of sample value unbiased estimate of the sigma in all times. S is unbiased: S = n/(n-1) * s, where s is std err from sample. $\endgroup$
    – Gregory Stelmashenko
    Aug 26, 2021 at 16:33
  • 3
    $\begingroup$ How do you figure that capital $S$ is unbiased? By Jensen's inequality, taking the square root of an unbiased estimator introduces bias. $\endgroup$
    – Dave
    Aug 26, 2021 at 16:34
  • $\begingroup$ Simple proof you could find in this book: Fetsje Bijma, Marianne Jonker, Aad van der Vaart, 2016 An Introduction to Mathematical Statistic, p 52 $\endgroup$
    – Gregory Stelmashenko
    Aug 26, 2021 at 16:39
  • 2
    $\begingroup$ I doubt that these authors can prove something that isn't true. @Dave has the key point here. $\endgroup$
    – Nick Cox
    Aug 26, 2021 at 17:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.