16
$\begingroup$

I was reading on the computation of the unbiased estimation of standard deviation and the source I read stated

(...) except in some important situations, the task has little relevance to applications of statistics since its need is avoided by standard procedures, such as the use of significance tests and confidence intervals, or by using Bayesian analysis.

I was wondering if anyone could elucidate the reasoning behind this statement, for example doesn't the confidence interval use the standard deviation as part of the calculation? Therefore, wouldn't the confidence intervals be affected by a biased standard deviation?

EDIT:

Thanks for the answers so far, but I'm not quite sure I follow some of the reasoning for them so I'll add a very simple example. The point is that if the source is correct, then then something is wrong from my conclusion to the example and I would like someone to point how how the p-value doesn't depend on the standard deviation.

Suppose a researcher wished to test whether the mean score of fifth graders on a test in his or her city differed from the national mean of 76 with a significance level of 0.05. The researcher randomly sampled the scores of 20 students. The sample mean was 80.85 with a sample standard deviation of 8.87. This means: t = (80.85-76)/(8.87/sqrt(20)) = 2.44. A t-table is then used to calculate that the two-tailed probability value of a t of 2.44 with 19 df is 0.025. This is below our significance level of 0.05 so we reject the null hypothesis.

So in this example, wouldn't the p-value (and maybe your conclusion) change depending on how you estimated your sample standard deviation?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ This does seem strange, for the reason you give. Perhaps you could give us the paragraph before as well in case there's something we're missing? One thing that makes the bias not a big deal is that it becomes pretty unimportant as the sample size gets bigger, and is probably not material compared to all the other problems eg model mis-specification that we normally have - but this is not the reason given in your source. $\endgroup$
    – Peter Ellis
    Commented Jul 28, 2012 at 6:05
  • 2
    $\begingroup$ @PeterEllis, this is actually from the wikipedia page on "unbiased estimation of standard deviation" (en.wikipedia.org/wiki/Unbiased_estimation_of_standard_deviation). $\endgroup$
    – BYS2
    Commented Jul 28, 2012 at 8:57

3 Answers 3

Reset to default
17
$\begingroup$

I agree with Glen_b on this. Maybe I can add a few words to make the point even clearer. If data come from a normal distribution (iid situation) with an unknown variance the t statistic is the pivotal quantity used to generate confidence intervals and do hypothesis testing. The only thing that matters for that inference is its distribution under the null hypothesis (for determining the critical value) and under the alternative (to determine power and sample). Those are the central and noncentral t distributions, respectively. Now considering for a moment the one sample problem, the t test even has optimal properties as a test for the mean of a normal distribution. Now the sample variance is an unbiased estimator of the population variance but its square root is a BIASED estimator of the population standard deviation. It doesn't matter that this BIASED estimator enters in the denominator of the pivotal quantity. Now it does play a role in that it is a consistent estimator. That is what allows the t distribution to approach the standard normal as the sample size goes to infinity. But being biased for any fixed $n$ does not affect the nice properties of the test.

In my opinion unbiasedness is overemphasized in introductory statistics classes. Accuracy and consistency of estimators are the real properties that deserve emphasis.

For other problems where parametric or nonparametric methods are applied, an estimate of standard deviation does not even enter into the formula.

Improve this answer
$\endgroup$
12
  • 8
    $\begingroup$ It does depend on the estimate but there is only one estimate for which the t with 19 degrees of freedom applies and that estimate is the square root of the usual estimate of the sample variance. If you use a different estimate of the standard deviation you have a different reference distribution for the test statistic under the null hypothesis. It is not the t. $\endgroup$
    – Michael R. Chernick
    Commented Jul 28, 2012 at 13:36
  • 3
    $\begingroup$ @BYS2: Note that in terms of the interval constructed in the example you give, nothing changes by multiplying the sample standard deviation by a scale factor (e.g., to make it unbiased). The distribution of the test statistic would change (slightly) in this case, but the CI constructed would end up being exactly the same! Now, if you did some "correction" that depended on the data themselves, that would yield something different (in general). See my comment under Glen's answer. $\endgroup$
    – cardinal
    Commented Jul 28, 2012 at 14:37
  • 4
    $\begingroup$ @BYS2: In the normal model case using the $t$-statistic, there is a nice correspondence between CIs and $p$-value. So, the $p$-value will not change if you "rescale" the sample standard deviation by a known constant. For example: Let $\tilde T_b = (\bar X - \mu)/(b\cdot \hat \sigma) = T / b$ for fixed $b > 0$. Then, $$\mathbb P(\tilde T_b > u) = \mathbb P(T > b u)$$ and so the critical value $\tilde t_{b,\alpha} = b t_\alpha$, i.e., there is a one-to-one correspondence between them. Does that make sense? $\endgroup$
    – cardinal
    Commented Jul 28, 2012 at 15:24
  • 1
    $\begingroup$ No what Cardinal is correctly pointing out is that it is possible to multiply the t statistic by a constant to essentially use a different estimate of standard deviation. The test statistic no longer has the t distribution. It is a slightly different distribution because of the constant. The mean changes by a factor of b and so does the standard deviation. When you go about computing the critical value for the test statistic it changes appropriately as he demonstrates above. $\endgroup$
    – Michael R. Chernick
    Commented Jul 28, 2012 at 19:35
  • 1
    $\begingroup$ @BYS2 Yes that's right. $\endgroup$
    – Michael R. Chernick
    Commented Jul 29, 2012 at 13:04
5
$\begingroup$

Consider an interval calculated on the basis of a pivotal quantity, like a t-statistic. The mean value of the estimator for standard deviation doesn't really come into it - the interval is based on the distribution of the statistic. So the statement is right as far as that goes.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Yes but doesn't the distribution of the statistic rely on its standard deviation which is unknown in most cases so you need to use an estimator? $\endgroup$
    – BYS2
    Commented Jul 28, 2012 at 6:24
  • 5
    $\begingroup$ (+1) Glen. To @BYS2: There are a couple of key points here. First, if we have a pivotal quantity at hand, it provides a very convenient means for constructing confidence sets, but they don't often exist. The whole point of a pivotal quantity is that the distribution depends purely on known quantities. Second, the pivotal quantity is intimately tied to the underlying model. If the data deviate from the assumed model, then the distribution of the test statistic may as well and its characterization as a pivotal quantity may not be quite as relevant. :) $\endgroup$
    – cardinal
    Commented Jul 28, 2012 at 14:41
4
$\begingroup$

Interpretation is always part speculation, but I think the implied meaning is that often you can get the result you want without estimating the standard deviation explicitly. In other words, I think the author is referring to situations where you would use no estimate of the standard deviation, rather than a biased estimate.

For instance, if you can construct an estimate of the whole distribution of a statistic, you can compute confidence intervals without using the standard deviation. In fact, for many (non-normal) distributions the standard deviation itself (and the mean) is not sufficient to compute an estimate of the confidence interval. In other cases, such as a sign test, you do not need an estimate for the standard deviation either.

(Of course, it is non-trivial to construct an unbiased estimate of a full distribution, and in Bayesian statistics it is actually quite common to introduce bias explicitly through the prior.)

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ It might be interesting to expand a little more fully on what you meant by the last paragraph. For example, if I can sample from the distribution of the statistic at hand, then the empirical cdf provides a very easy, simple means for generating a pointwise unbiased estimate of the distribution function. :) $\endgroup$
    – cardinal
    Commented Jul 28, 2012 at 14:48
  • 1
    $\begingroup$ @cardinal True, but this assumes that you can sample from the distribution of the statistic. This is not always possible. For instance, consider the statistic $\max_i X_i$. It turns out it is impossible to construct an unbiased estimator for $\max_i X_i$, even if we can obtain unbiased samples for each $X_i$. $\endgroup$
    – MLS
    Commented Jul 28, 2012 at 15:49
  • 1
    $\begingroup$ To be clear: $X_i$ is just a random variable, and I assumed that $i$ can take at least 2 different values (i.e., there are at least two variables). Otherwise, unbiased estimates for $\max_i X_i$ are not so hard to construct :) $\endgroup$
    – MLS
    Commented Jul 28, 2012 at 15:50
  • 2
    $\begingroup$ This is true and close to the point I was trying to draw out. The first sentence of the last paragraph is referring to constructing an unbiased estimate of a nonlinear statistical functional from, e.g., a single random sample. This is quite different from constructing an unbiased estimate of a full distribution from a random sample of the function itself. :-) $\endgroup$
    – cardinal
    Commented Jul 28, 2012 at 16:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.