17
$\begingroup$

A standard error is the estimated standard deviation $\hat \sigma(\hat\theta)$ of an estimator $\hat\theta$ for a parameter $\theta$.

Why is the estimated standard deviation of the residuals called "residual standard error" (e.g., in the output of R's summary.lm function) and not "residual standard deviation"? What parameter estimate do we equip with a standard error here?

Do we consider each residual as an estimator for "its" error term and estimate the "pooled" standard error of all these estimators?

Improve this question
$\endgroup$
7
  • 6
    $\begingroup$ I think that's an R thing. I don't think other software necessarily uses that phrasing, & 'residual standard deviation' is common in textbooks, eg. I don't have an answer, but I always thought it was weird that R uses that phrase. $\endgroup$ – gung - Reinstate Monica Apr 1 '15 at 20:00
  • $\begingroup$ @gung: that could be the explanation! When googling "residual standard error" in quotes I get only 0.1% of the hits than without quotes... $\endgroup$ – Michael M Apr 1 '15 at 20:03
  • $\begingroup$ I could put that as a (non-)answer, if you'd prefer. $\endgroup$ – gung - Reinstate Monica Apr 1 '15 at 20:05
  • 1
    $\begingroup$ @gung it's funny how using specific software shapes your thinking: I'd never call it "residual sd" - residuals are not data but errors, so residual error seems to be proper name. But if you think about it it really seems an R-thing. $\endgroup$ – Tim♦ Apr 1 '15 at 20:09
  • 2
    $\begingroup$ @Tim, it might correctly be considered an estimate of the standard deviation of the errors, but the residuals are not technically the errors themselves. Nor is it the standard error of the error SD, for what that's worth. $\endgroup$ – gung - Reinstate Monica Apr 1 '15 at 20:17
13
$\begingroup$

I think that phrasing is specific to R's summary.lm() output. Notice that the underlying value is actually called "sigma" (summary.lm()$sigma). I don't think other software necessarily uses that name for the standard deviation of the residuals. In addition, the phrasing 'residual standard deviation' is common in textbooks, for instance. I don't know how that came to be the phrasing used in R's summary.lm() output, but I always thought it was weird.

Improve this answer
$\endgroup$
3
  • $\begingroup$ How is summary.lm(reg)$sigma different from sd(reg$residuals)? $\endgroup$ – airstrike Feb 18 '16 at 6:23
  • 3
    $\begingroup$ @AndréTerra, the correct degrees of freedom is n - p, which is what summary uses. sd uses var which uses n - 1 degrees of freedom. If you manually compute the standard deviation of the residuals dividing by n - p then you will get the same answer as what summary provides. $\endgroup$ – Jdub Sep 15 '16 at 17:04
  • 3
    $\begingroup$ To corroborate gung, I cite from the R documentation of stats::sigma: The misnomer “Residual standard error” has been part of too many R (and S) outputs to be easily changed there. $\endgroup$ – NRH Oct 5 '16 at 20:25
3
$\begingroup$

From my econometrics training, it is called "residual standard error" because it is an estimate of the actual "residual standard deviation". See this related question that corroborates this terminology.

A Google search for the term residual standard error also shows up a lot of hits, so it is by no means an R oddity. I tried both terms with quotes, and both show up roughly 60,000 times.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Interesting. But why would you call an estimate of a standard deviation of any random variable (like an error term; and not a specific estimator) a "standard error"? $\endgroup$ – Michael M Apr 2 '15 at 6:41
  • $\begingroup$ My thinking is we need to have a name for the estimate (to distinguish from the actual value), any name is as good as another. But surely someone more knowledgeable about the etymology can offer a better reason. Note that there is definitely a parallel with the coefficient standard error, which is the estimate of the coefficient estimate 's standard deviation. $\endgroup$ – Heisenberg Apr 2 '15 at 15:11
0
$\begingroup$

Put simply, the standard error of the sample is an estimate of how far the sample mean is likely to be from the population mean, whereas the standard deviation of the sample is the degree to which individuals within the sample differ from the sample mean.

Standard error - Wikipedia, the free encyclopedia

Improve this answer
$\endgroup$
1
  • 6
    $\begingroup$ This is true, but does not actually answer the question. What R calls the "residual standard error" is not "an estimate of how far the sample mean is likely to be from the population mean". $\endgroup$ – gung - Reinstate Monica Apr 1 '15 at 20:03
0
$\begingroup$

A fitted regression model uses the parameters to generate point estimate predictions which are the means of observed responses if you were to replicate the study with the same XX values an infinite number of times (when the linear model is true).

The difference between these predicted values and the ones used to fit the model are called "Residuals" which, when replicating the data collection process, have properties of random variables with 0 means. The observed residuals are then used to subsequently estimate the variability in these values and to estimate the sampling distribution of the parameters.

Note:

When the residual standard error is exactly 0 then the model fits the data perfectly (likely due to overfitting).

If the residual standard error can not be shown to be significantly different from the variability in the unconditional response, then there is little evidence to suggest the linear model has any predictive ability.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Good attempt at explaining it but this story is not compatible with the definition of the standard error. The standard error is the standard deviation of the sampling distribution of the mean, but the distribution of the residuals is not a sampling distribution. Specifically, the standard error depends on N but the standard deviation of the residuals does not. $\endgroup$ – tmalsburg Oct 22 '20 at 17:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.