5
$\begingroup$

This question addresses calculating a p value from the mean and standard deviation statistics of a sample. I understand that the -general- philosophy is to divide the sample standard deviation by the root of the sample size to get the standard deviation of the sampling distribution, using the assumption that the standard deviation of the sample is roughly equal to the standard deviation of the hypothetical larger population. Then one calculates a z-score from the number of standard deviations of the sampling distribution to calculate what percent of the time the observed result would have occurred by random chance. I understand that the particular formula for the sampling distribution standard deviation depends on the particular statistic, say difference of means is a different formula.

The texts and videos that I've looked at use language like "the sample standard deviation is the best number we have available to estimate the population standard deviation." I just don't find that explanation satisfying.

This approach hinges on the validity of estimating the standard deviation of the entire population as being approximately equal to the standard deviation of the representative sample. However, we don't make the same assumption that the mean of the population is the approximately equal to the mean of the sample. At some level, it feels like the final result of significance or non-significance is only self-validating or checking for self-consistency of an assumption that is baked into the methodology.

So to restate, why is the sample standard deviation a good approximation of the population standard deviation, but the sample mean is not a good approximation of the population mean? I found online an equation for standard deviation of the sampling distribution of standard deviations:

standard error of standard deviation = .71 sample standard deviation / root N.

Does the relative narrowness of the standard error compared to standard deviation play a role in justifying the approximation?

Thank you

Improve this question
$\endgroup$
6
  • 7
    $\begingroup$ "why is the sample standard deviation a good approximation of the population standard deviation, but the sample mean is not a good approximation of the population mean?" Where have you seen it asserted that the sample mean is not a good approximation of the population mean? It is often used as an estimator of a population mean. $\endgroup$
    – Michael Hardy
    Jun 16, 2019 at 1:16
  • $\begingroup$ I've wondered the same thing. We may consider the sample mean the best available estimate of the population mean. But we know it's not perfect, so we want to quantify our uncertainty with a p-value or CI. So we do so by estimating the standard error, with the very same data we don't entirely "trust". $\endgroup$
    – user3391564
    Jun 16, 2019 at 1:31
  • $\begingroup$ @Michael When a class shifts from teaching a z test with population standard deviation known to a t test using the sample standard deviation, sometimes unfortunate language like this is used that creates a seemingly illogical inconsistency between how the mean and standard deviation are described and used. The treatment of the sd as a nuisance parameter when testing a mean seems to further the confusion. Some of the handwaving done to transition from the ztest to the ttest without getting into technical details likely leads to confusion as well. $\endgroup$
    – jsk
    Jun 16, 2019 at 1:55
  • $\begingroup$ “I just don’t find that satisfying”. It is not satisfying, indeed. The problem is that the text you’re using is not advanced enough to formally justify/prove the statement. You will find proofs for statements like those in mathematical statistics textbooks. Check, for example, Statistical Inference by Casella & Berger. $\endgroup$
    – Sigma
    Jun 16, 2019 at 4:24
  • $\begingroup$ You wrote "However, we don't make the same assumption that the mean of the population is the approximately equal to the mean of the sample." That is simply not true. If you could tell us what caused you to think we do not use the sample mean to estimate the population mean, maybe your question could be understood. $\endgroup$
    – Michael Hardy
    Jun 16, 2019 at 16:25

2 Answers 2

Reset to default
1
$\begingroup$

I'm answering my own question but I think that I have an intuitive justification for why the sample standard deviation reflects the population standard deviation.

As our sample size becomes larger and larger, it better reflects the population, and thus the variation of the population. I realize that we could say the same for the sample mean, but at least for the case of flipping coins, the sample s.d. seems to tend adhere more towards the pop s.d. compared to sample mean.

Suppose we have a true 50 50 coin: p=0.5 q=0.5. Half of all flips in the coin's history (q = 0.5) give tails = 0, and half of all flips in the coin's history (p = 0.5) give heads = 1. The population mean of all flips in coin's history = 0.5, and Standard Deviation = root (0.5*0.5) = 0.5.

if flip 10 times and get q=0.6, p=0.4 for 1 sample then sample mean = 0.4, SD = root (0.6 x 0.4) = 0.490 sample mean has decreased 20% from pop mean, but SD decreased 2%

if flip 10 times and get q=0.7, p=0.3 for 1 sample then sample mean = 0.3, SD = root (0.7 x 0.3) = 0.458 sample mean has decreased 40% from pop mean, but SD decreased 10%

if flip 10 times and get q=0.8, p=0.2 for 1 sample then sample mean = 0.2, SD = root (0.8 x 0.2) = 0.4 sample mean has decreased 60% from pop mean, but SD decreased 20%

suppose flip 10 times and get q=0.9 p = 0.1 for 1 sample then sample mean = 0.1, SD = root (0.9 * 0.1) = 0.3 sample mean has decreased 80% from pop mean, but SD decreased 40%

This scenario shows that over the vast majority of the sampling distribution of possible sample means of 10 flip outcomes, the sample standard deviation does not stray too far from 0.5, even though the sample mean can vary much more

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ This is not a good example of a general behavior because the possible outcomes are so limited. More generally, the variation in the sample mean has a magnitude proportional to the square root of a second moment while the variation in the sample variance has a magnitude proportional to the square root of a fourth moment. It therefore is essential to consider the magnitudes of the first, second, and fourth moments of the underlying distribution. $\endgroup$
    – whuber ♦
    Nov 2, 2020 at 14:36
  • 1
    $\begingroup$ Thank you for the comment, that is helpful to me. $\endgroup$
    – lamplamp
    Nov 4, 2020 at 20:15
-3
$\begingroup$

No matter the shape of the population distribution, the Central Limit Theorem holds that the sampling distribution of the sample means (and sample variances) approaches a normal distribution as the sample size $T$ (number of observations or data points) gets larger, especially for sample sizes over 30. As you take more samples, the sample distribution graph will look more like a normal distribution, and so on average, the sample moments will be the population moments as $T\uparrow$. (If you add up the means or standard deviations from all of your samples and find the average, that average will be close to the actual moments for the population.)

Improve this answer
$\endgroup$
2
  • $\begingroup$ You wrote "as $T\uparrow$", but you never said what "$T$" is. $\qquad$ $\endgroup$
    – Michael Hardy
    Jun 16, 2019 at 16:28
  • 2
    $\begingroup$ The conclusion about the "sample distribution graph" is incorrect. That's not what the CLT says and the world abounds with counterexamples: as the sample size increases, the empirical distribution of the sample approximates the underlying distribution, which is rarely Normal. $\endgroup$
    – whuber ♦
    Jul 29, 2020 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.