I have this code here (it's out of a book called "An Introduction to Bootstrap Methods with Applications to R")

We are working with the estimator: $S_n^2=\frac{\sum_{i=1}^{n}(X_i-X_b)^2}{n}$ where $X_b = \frac{\sum_{i=1}^{n}X_i}{n}$, $S_n^2$ is the maximum likelihood estimate of $\sigma^2$. I'm relatively new to this statistics regarding likelihoods and estimators.

x<-rnorm(n) # random sample of unit normal variate
varx<-var(x)*(n-1)/n  # sample variance, uncorrected
c(varx, varx-1.0, -1/sqrt(n)) # sample variance and bias relative to true 
                      # value 1.0 and expected value of bias

and I'm having trouble comprehending the last argument of the last line.


Can someone tell me how he got this term that he calls "expected value of bias"?

New edited below

In the book he says that the estimator $S_n^2$ has a bias of $b=-\sigma^2/n$, which I think the -1/sqrt(n) is the expected value of this value $b$. I will have to go and research how the bias of that estimator comes about, however if you have anything to enlighten me on this, it would be greatly appreciated.

I'm familiar with with we divide by (n-1) when estimating population variance from an intuitive and mathematical perspective. He is demonstrating the importance of this by biasing the output of var().

I believe my question is even more simplistic than this, in that I cannot see the connection of -1/sqrt(n) to this. I think it is a basic math thing that I'm not seeing.

enter image description here

  • 1
    $\begingroup$ Possible duplicate of Intuitive explanation for dividing by $n-1$ when calculating standard deviation? $\endgroup$ – jbowman Oct 3 '18 at 0:37
  • 1
    $\begingroup$ The $-1/\sqrt{n}$ isn't the expected value of the bias when $\sigma = 1$, it is the bias when $\sigma = 1$ (as it does in this case.) $\endgroup$ – jbowman Oct 3 '18 at 0:38
  • 1
    $\begingroup$ @Bucephalus In your comment you say "$b=-\sigma^2/n=-\sigma/\sqrt{n} $". How does that second equality work? $\endgroup$ – Glen_b Oct 3 '18 at 1:38
  • 1
    $\begingroup$ The problem with your algebra there isn't simply one of taking the square root of a negative number. You have stated $-k^2=-k$ (for a particular $k$), and irrespective of whether $k$ was positive, you don't get to manipulate powers like that and still claim to have equality. "$=$" isn't shorthand for "the next thing I thought about" -- it's a statement that two things are the same. $-k^2$ and $-k$ are not interchangeable quantities. $\endgroup$ – Glen_b Oct 3 '18 at 2:09
  • 1
    $\begingroup$ The same problem again (without the minus sign), how can you say "$\sigma^2/n = \sigma/\sqrt{n}$? What sort of thinking says a quantity is equal to its square? $\endgroup$ – Glen_b Oct 3 '18 at 2:15

The bias of $S_n^2$ as an estimate of variance is indeed $-\sigma^2/n$, as given in the text, & as you can confirm from e.g. Wikipedia. You probably have a early printing with a typo: the version currently on Google Books gives the code

c(varx, varx-1.0, -1/n) # sample variance and bias relative to true value 1.0 and expected value of bias

NB The terminology in the comment could be improved by substituting "error" for "bias": the difference between the estimate of the variance & its true value is the error & the expected value of the error its bias. (The "expected value of the bias" would just be the bias, as there are no random variables in the expression: $\operatorname{E}( -\sigma^2/n)=-\sigma^2/n$.)

| cite | improve this answer | |
  • $\begingroup$ That's very helpful @Scortchi. You're right, I just went and checked the book on safaribooks, and I have edited nto my post a screenshot of the code snippeted straight out of the book now. Good pickup. $\endgroup$ – Bucephalus Oct 3 '18 at 11:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.