0
$\begingroup$

I'm learning applications on Central Limit Theorem and got really confused with a few points. According to this tutorial, the procedure to apply CLT usually goes like this: enter image description here

So if SD is the population standard deviation, how are we gonna get it?? Isn't the whole population standard deviation what we eventually calculate by applying CLT and analyzing a sample of the whole population? How come the population standard deviation become a prerequisite??

Please tell me this tutorial is wrong.

I think the SD actually refers to the standard deviation of a sample (of some size n), which we can actually get easily. For example:

  1. sample the whole population with a sample size of n (e.g. randomly select 10000 users from the whole population of 10 billions)
  2. calculate the mean of the 10000 measurements.
  3. calculate SE = SD / sqrt(10000), where SD is the standard deviation of the 10000 measurements, instead of the standard deviation of the whole population of 10 billions.

This explanation would make much more sense.

Any thoughts?

$\endgroup$
2
  • 1
    $\begingroup$ (a) IMHO this part of the tutorial (even taken out of its context) seems to be poorly written. (b) The CLT requires existence of population mean $\mu$ and population variance $\sigma.$ Then even if the population is not normal, for large enough $n,$ the sample mean $\bar X$ is approximately normal with $E(\bar X) = \mu$ and $SE(\bar X) = \sigma/\sqrt{n}.$ (c) If you don't know $\sigma$ and approximate it with sample SD $S,$ then you need to use methods involving Student's t distribution. $\endgroup$
    – BruceET
    Commented Apr 26, 2019 at 6:13
  • $\begingroup$ @BruceET thanks for making it clear. Just to confirm, if you check out this question: stats.stackexchange.com/questions/405157/…, we never care about the standard deviation within one sample, right? $\endgroup$ Commented Apr 26, 2019 at 6:27

1 Answer 1

2
$\begingroup$

I have looked at your other two questions. You never make it entirely clear (to me anyhow) what your objective is. So, owing to your persistence, I will give you some examples, computations, and explanations that may be helpful. [Most of what is below is introduced in standard elementary textbooks on applied statistics. Such a textbook might be a better-organized guide than online tutorials and videos. Used copies of out-of-date editions of such books are pretty cheap on Amazon and elsewhere.]

Suppose you take a random sample of size $n = 1000$ and find sample mean $\bar X = 247$ and sample standard deviation $S = 87.$

Your goal might (1) be to find a 95% confidence interval (CI) for the population mean $\mu$, or (2) to test the null hypothesis $H_0: \mu = 250$ against the alternative $H_a: \mu < 250.$ Then (1) requires a t confidence interval and (2) requires a one-sample t test.

(1) The 95% CI for $\mu$ is of the form $\bar X \pm t^*S/\sqrt{n},$ where $t^* = 1.962$ cuts 2.5% from the upper tail of Student's t distribution with $n - 1 = 999$ degrees of freedom (which is very close to standard normal). My late-night computations, which you should verify, give the interval $(241.60, 252.40 ).$

In Minitab statistical software, the output for a t confidence interval procedure is shown below; it agrees with my computation.

One-Sample T

   N    Mean  StDev  SE Mean        95% CI
1000  247.00  87.00     2.75   (241.60, 252.40)

(2) The test statistic for the t test is $T = \frac{\bar X - \mu_0} {S/\sqrt{n}} =\frac{247-250}{87/\sqrt{1000}} = -1.090.$ For a left-sided t test specified by $H_0$ and $H_1$ at the 5% level of significance, the 'critical value' is $c = - 1.646.$ That is, you could reject $H_0$ at the 5% level if $T < -1.646,$ but $T = -1.090$ so you cannot reject. [The critical value cuts 5% from the lower tail of Student's t distribution with 999 degrees of freedom.]

The 'P-value' of the test is the probability under that distribution of a t statistic less than the observed $-1.090.$ You need some sort of software to find the P-value, which turns out to be $0.1380.$ Using the P-value as a criterion, you could reject $H_0$ if the P-value were smaller than 5% (which it is not).

The Minitab printout for this left-sided t test is shown below (slightly edited for relevance); it shows the same test statistic and P-value as in my computations above.

One-Sample T 

Test of μ = 250 vs < 250

   N    Mean  StDev  SE Mean       T      P
1000  247.00  87.00     2.75   -1.09  0.138

Note: In one of your questions you mentioned a sample of size $n = 10,000.$ If you had the same sample mean $\bar X = 247$ and sample SD $S = 87$ a sample of that size, then the 95% CI would be $(245.295, 248.705)$ and the P-value of the one-sided test would have been smaller than $0.0005,$ leading to rejection of the null hypothesis. Sample size matters.

Addendum about a CI for population SD: For normal data: Because $$Q=\frac{(n−1)S^2}{σ^2} \sim \mathsf{CHISQ}(\text{df} = n−1),$$ one can use a printed table of chi-squared distributions or software to find quantiles .025 and .975, $L$ and $U,$ respectively, of that distribution to get $$P(L<Q<U)= \cdots = P\left(\frac{(n−1)S^2}{U}<σ^2<\frac{(n−1)S^2}{L}\right)=.95.$$

[Notice the 'reversal' of $L$ and $U,$ which results from taking reciprocals in solving the inequality to 'isolate' $\sigma^2.]$

Hence a 95% CI for $σ^2$ is of the form $$\left(\frac{(n−1)S^2}{U},\,\frac{(n−1)S^2}{L}\right).$$

Take square roots of endpoints to get 95% CI for $σ.$

For example, if a sample of size $n=50$ from a normal population has sample variance $S^2=34.5,$ then a 95% CI for the population SD $σ$ is $(4.91,\,7.32.).$ [Notice that the point estimate $S=5.87$ is contained within this CI, but not at its midpoint (because chi-squared distribution is skewed.] Computation from R:

v = 34.5;  sqrt(49*v/qchisq(c(.975,.025), 49))
[1] 4.906476 7.319376

Output from Minitab:

95% Confidence Intervals

                CI for        CI for
 Method          StDev       Variance
 Chi-Square  (4.91, 7.32)  (24.1, 53.6)

As you suggest, such intervals tend to get shorter with increasing $n.$ However, intervals can still be disappointingly long, even for moderately large $n.$ If the sample variance $S^2 = 34.5$ had resulted from a sample of size $n=500,$ then the resulting 95% CI for $\sigma$ would be $(5.531,\, 6.262).$ "Variances are very variable."

v = 34.5;  sqrt(499*v/qchisq(c(.975,.025), 499))
[1] 5.530786 6.262223
$\endgroup$
6
  • $\begingroup$ Thank you so much! To clarify, my confusion is on the comparison between the SD of whole population vs SD of one sample (size 1000 in your answer). e.g. Can we use SD of one sample to calculate the SE of samples mean? In your answer you made it clear that we use SD of one sample divided by sqrt(n) to calculate SE. In many other sources, they use SD of the whole population. So I was confused: how accurately can SD of one sample approximate the SD of whole population?? $\endgroup$ Commented Apr 26, 2019 at 15:36
  • 1
    $\begingroup$ In both of my examples (CI & t test) $S/\sqrt{n}$ is the estimated standard error of the sample mean $\bar X.$ People often call this the 'standard error', omitting the word estimated when the estimation is thought to be obvious. $\endgroup$
    – BruceET
    Commented Apr 26, 2019 at 15:43
  • $\begingroup$ This makes much more sense now: SD of one sample / sqrt(n) = estimated SE of sample mean; SD of whole population / sqrt(n) = exact SE of sample mean. An important follow-up questions is: How accurately can we approximate SD of whole population using SD of one sample? Of course the bigger the sample size, the better the approximation. But it must require some strict mathematics to support this approximation. Is there any? $\endgroup$ Commented Apr 26, 2019 at 15:52
  • $\begingroup$ The former comment in response to your last question was deleted here and put into an Addendum to my Answer. (Computations shown, equations displayed, abbreviations spelled out, articles supplied.) $\endgroup$
    – BruceET
    Commented Apr 26, 2019 at 19:16
  • $\begingroup$ I'm a newbie in using this website. what does it mean? ((Computations shown, equations displayed, abbreviations spelled out, articles supplied.) ) $\endgroup$ Commented Apr 27, 2019 at 3:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.