0
$\begingroup$

I am reading the book " An Introduction to Statistical Learning " and I have trouble understanding their explanation of RSE(Residual Standard Error ) . This is what the book says :

"The RSE is an estimate of the standard deviation of \epsilon . Roughly speaking, it is the average amount that the response will deviate from the true regression line. It is computed using the formula :"

enter image description here

What I don't understand here is definition of RSE as " Standard deviation of \epsilon " .

If Residual = y(i) - yHat(i)

and going by my assumption that \epsilon is also = y(i) - yHat(i)

Isn't the formula of RSE just computing the "root(squared mean)" of \epsilon ?

If \epsilon = y(i) - yHat(i) ,

Then , standard deviation of \epsilon would be sum[y(i) - yHat(i) - mean(Y - Yhat)] / n-2 which is not what the above formula does .

So , Technically , I think RSE is just squared mean of the Residuals or \epsilon and would be wrong to call it " Standard deviation of \epsilon " , if we go by the actual formula for standard deviation .

Or , In my opinion this should mean the standard deviation of predicted yHat values rather than \epsilon .

So , is this a misuse of the term " standard deviation " in the book or Am I missing something ?

Please correct or help me understand .

$\endgroup$
1
$\begingroup$

I believe this is because in OLS the average of residuals is $0$. Hence the formula for standard deviation of residuals ($\epsilon)$ can be given as:

$$\sqrt\frac{\sum (\epsilon_i - \mu_{\epsilon})^2}{n-2}$$

but since in OLS $\mu_{\epsilon}=0$ by construction (see explanation of why this is so in this Mathematics.SE answer) you are left with:

$$\sqrt\frac{\sum (\epsilon_i - 0)^2}{n-2}= \sqrt\frac{\sum (\epsilon_i)^2}{n-2} \implies \sqrt\frac{\sum (y_i - \hat{y_i})^2}{n-2}$$

since $\epsilon_i=y_i - \hat{y_i}$

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Just what I needed . Great Answer ! $\endgroup$ – Bharathi Aug 29 at 14:11
  • $\begingroup$ @Bharathi you are welcome I also encourage you to read the other answer about difference between residuals and unobservable disturbance term since that’s good caveat $\endgroup$ – 1muflon1 Aug 29 at 14:14
1
$\begingroup$

Let's specify that we are speaking about simple linear regression, $y_i=\beta_0+\beta_1x_i+\epsilon_i$.

If Residual = y(i) - yHat(i) and going by my assumption that \epsilon is also = y(i) - yHat(i)

The error is $\epsilon_i=y_i-\beta_0-\beta_1x_i$, the residual is $e_i=y_i-\hat\beta_0-\hat\beta_1x_i=y_i-\hat{y}_i$. They are not equal.

Isn't the formula of RSE just computing the "root(squared mean)" of \epsilon ?

You can't compute the root squared mean of $\epsilon$ because you can't observe $\epsilon$. In other words, you don't know, and will never know, $\beta_0$ and $\beta_1$, so you can't calculate $y_i-\beta_0-\beta_1x_i$.

What I don't understand here is definition of RSE as " Standard deviation of \epsilon " .

Pay attention! Your book says that "The RSE is an estimate of the standard deviation of $\epsilon$": since you can't observe $\epsilon$, the best you can do is using residuals to get an estimate of errors' standard deviation.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.