2
$\begingroup$

so i was reading statistics by Freedman and in the section where he describes the t-test he goes on to explain why we would use the SD+ instead of the regular SD to calculate the S.E with the sample being really small and all. what i don't get is the formula for the SD+ itself. in the book the formula for SD+ is given as: the SD multiplied by the square root of the ratio of the sample size to the sample size - 1 i.e [ sqr{n/(n-1)}] x SD. what does this mean? and why do we do this? is this anything like using the correction factor? also, googling SD+ doesn't yield any results. is it more commonly referred to by some other name?

Thanks

$\endgroup$
4
  • 1
    $\begingroup$ Please beware of the fact that not everyone is familiar with the same conventions and may not understand what you're referring to as "SD" and "SD+" unless they have the same text as you do in front of their eyes $\endgroup$
    – David
    Jul 3, 2019 at 15:14
  • $\begingroup$ I think you just need to look up “Bessel’s correction” in wiki. And here at stats.SE as well. Hope this helps! $\endgroup$
    – Ed V
    Jul 3, 2019 at 16:16
  • 1
    $\begingroup$ @David i know, which is why i asked in the last part of the post about how I can't find it anywhere else and whether it is conventionally called something else. $\endgroup$
    – NoLifeKing
    Jul 4, 2019 at 6:35
  • $\begingroup$ @EdV yeah this seems to be what i was looking for. THnaks $\endgroup$
    – NoLifeKing
    Jul 4, 2019 at 6:37

1 Answer 1

2
$\begingroup$

I will try to put the distinction between the two estimators of the standard deviation into perspective. At least at the start, I'll try to stay at the approximate level of Freedman's book.

The so-called 'maximum likelihood estimator' of the population variance $\sigma^2$ (assuming the population mean is unknown and estimated by $\bar X)$ is $V = \frac 1n\sum_{i=1}^n (X_i - \bar X)^2.$ Unfortunately, $V$ is a biased estimator: that is, $E(V) < \sigma^2.$ Bessel's correction is to use $$S^2 = \frac{1}{n-1}\sum_{i=1}^n (X_i - \bar X)^2 = \frac{n}{n-1}V,$$ which is unbiased: $E(S^2) = \sigma^2.$

Let's see what happens if we use R to take a million samples of size $n=10$ from a population with variance $\sigma^2 = 1,$ find the million sample variances $S^2,$ and average them. The result is $1,$ within the margin of simulation error. [So you won't get the idea that the population has to be normal for this to work, I used the population $\mathsf{Unif}(0,\sqrt{12}),$ which has $\sigma^2.$ Also notice that the function var in R is programmed to use $n-1$ in the denominator.]

set.seed(1776)
m = 10^6;  n = 10
s.sq = replicate(m, var(runif(n, 0, sqrt(12))))
mean(s.sq)
[1] 1.000178       # aprx E(S^2) = 1
 v = (n-1)*s.sq/n
mean(v)
[1] 0.9001602      # aprx E(V) = 0.9

Even though $E(S^2) = \sigma^2,$ it turns out that $E(S) < \sigma.$ The discrepancy between $E(S)$ and $\sigma$ depends on the sample size and the type of population distribution involved. (Often, the discrepancy is very small for large samples.)

Adding one more step to the simulation above, we see that for samples of size $n = 10$ from $\mathsf{Unif}(0,\sqrt{12}),$ we have $E(S) \approx 0.986\sigma.$ For a normal sample of size $10,$ we have $E(S) \approx 0.973.$ [Ref.]

s = sqrt(s.sq);  mean(sqrt(s)) 
[1] 0.9864858
sqrt(2/9)*gamma(10/2)/gamma(9/2)  # normal formula [see ref.]
[1] 0.9726593

There is some controversy whether to use $S^2$ or $V$ to estimate $\sigma^2.$ There are pedagogical, historical, and theoretical advantages on both sides. Overall, I think it is fair to say that more textbooks and statistical software programs use $S^2$ than use $V.$

  • In favor of $V:$ It can be messy to explain Bessel's correction to beginning students. According to various criteria, for normal data $V$ can give slightly better estimates. [I give a related simulation below.]

  • In favor of $S^2:$ formulas for many tests on normal data use $n-1$ in the denominator. Many statistical software programs use $S^2$ and related theoretical formulas. Student's t distribution used in t tests has $n-1$ degrees of freedom and the usual formulas for $T$-statistics and t confidence intervals use $S^2.$ Similarly, $S^2$ and the chi-squared distribution with $n-1$ degrees of freedom are used for tests and confidence intervals for $\sigma^2.$ [That may explain the reference to t tests in your Question.]

One criterion for judging the 'goodness' of an estimator is 'mean square error'. If $T$ is an estimator of $\tau,$ then $$MSE(T,\tau) = E[(T - \tau)^2] = Var(T) + B_{T,\tau}^2,$$ where $B_{T,\tau} = E(T - \tau)$ is the 'bias'. Sometimes, it is more convenient to use 'root mean square error` $RMSE(T,\tau) = \sqrt{MSE(T,\tau)}.$ in order to use the original units of the data instead of squared units.

For normal data, the following simulation illustrates that $RMSE(V, \sigma^2) < RMSE(S^2, \sigma^2).$ We use samples of size $n = 10$ from $\mathsf{Norm}(100, 15),$ so that $\sigma^2 = 225.$

set.seed(2019)
m = 10^6;  n = 10
s.sq = replicate( m, var(rnorm(n, 100, 15)) )
mean(s.sq)
[1] 224.9544                # aprx 225
sqrt(mean((s.sq - 225)^2))
[1] 106.0162                # aprx RMSE(S^2) > RMSE(V)                       
v = (n-1)*s.sq/n
mean(v)
[1] 202.459                 # biased estimator
sqrt(mean((v - 225)^2))
[1] 98.04103                # aprx RMSE(V) < RMSE(S^2)

enter image description here

Why doesn't this settle the issue so that everyone adopts $V$ over $S^2?$ A few reasons:

  • Some practitioners object to underestimates of $\sigma^2.$
  • There are criteria other than RMSE, which some feel tends to put unduly heavy emphasis on large errors.
  • Slippery slope: Using $n+1$ in the denominator of the variance estimator gives an even smaller RMSE, but then underestimating $\sigma^2$ becomes even more serious. [However, using $n+2$ in the denominator increases RMSE over using $n+1.]$
  • Inertia.
  • Perhaps others to be mentioned in comments.
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.