4
$\begingroup$

From what I've learnt, standard error is a concept related to sampling distribution. So why is the term 'residual standard error' used to refer to $\sqrt{\frac{RSS}{n-2}}$ and not $\sqrt{E(r_i - E(r_i))^2}$ in SLR context?

(I mentioned Simple Linear Regression only because I have not learnt other regression techniques yet)

|cite|improve this question
$\endgroup$
1
$\begingroup$

The residual standard error in this context is the square root of the estimated variance of the error term. In a linear regression with an intercept term and a single explanatory variable we have:

$$\hat{\mathbb{V}}(\varepsilon) = \text{MSE} = \frac{\text{RSS}}{\text{df}_\text{Res}} = \frac{\text{RSS}}{n-2}.$$

So the quantity you are referring to is $\hat{\mathbb{S}}(\varepsilon) = \sqrt{\text{RSS}/(n-2)}$. The other quantity you refer to is the standard deviation of the $i$th residual, which is not a fixed quantity over $i=1,...,n$ since it depends on the leverage values of the data points.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Well I used r_i here as a random variable and not a specific single residual value from a sample. So my question was like, since it is named 'standard error' I thought it must be something related to the sampling distribution of randome variable r_i, and not RSS/n-2 which is a statistic calculated from a specific sample (of n observation of r_i) $\endgroup$ – user8931048 Aug 9 '18 at 9:34
  • 1
    $\begingroup$ Each of the random variables $R_1,...,R_n$ have a different variance, which is affected by the leverage of each data point. That means that there is no single generic variance for the residual. $\endgroup$ – Reinstate Monica Aug 9 '18 at 9:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.