4
$\begingroup$

We know that in simple linear regression the variance of the regression error, $\sigma^2$, is estimated by $\frac {\sum_{i=1}^{n} (y_i - \hat y)^2} {n-2}$, i.e., the Mean Squared Error of the errors. But to standardize the residuals it is said to use the "standard error" of the residuals. Is this the exact same thing as that formula I just wrote?

Or is it $\frac{\sum_{i = 1}^{n} (\epsilon_i - \bar \epsilon)^2}{n-1}$, assuming $\epsilon_i$ is the $i$th residual?. This makes more sense to me. Are the two somehow equivalent?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ If all assumptions are fulfilled then $\bar{e}=0$ and as $y_i -\hat{y}_i=e_i$ they are equivalent $\endgroup$
    – user83346
    Commented Jun 16, 2016 at 5:23
  • $\begingroup$ @Wolfgang, to calculate $\sigma^2$, why we divide by $n-2$ instead of $n-1$? $\endgroup$
    – rsl
    Commented Jun 16, 2016 at 7:05
  • 1
    $\begingroup$ @Moazzem Hossen We divide by n-2 to get an unbias estimate of the variance $\endgroup$
    – Mauro Augusto
    Commented Jun 16, 2016 at 9:20
  • $\begingroup$ Do you use Weighted least squares? In thic case It's suppused that errors depend of regressors and have different variances. $\endgroup$
    – mmmmmmm
    Commented Dec 30, 2016 at 10:30
  • $\begingroup$ The term standard error has nothing to do with standardizing residuals. $\endgroup$
    – Nick Cox
    Commented Dec 30, 2016 at 10:54

1 Answer 1

Reset to default
2
$\begingroup$

The standardized residual is the residual divided by its standard deviation. Standardizing residual is a method for transforming data so that its mean is zero and standard deviation is one. If the distribution of the residuals is approximately normal, then $95\%$ of the standardized residuals should fall between $-2$ and $+2$. If many of the residuals fall outside of $+2$ or $–2$, then they could be considered unusual. However, about $5\%$ of the residuals could fall outside of this region due to chance. Consider the regression equation $\hat y_i = \beta_0 + \beta_1x_i + \epsilon_i$ and to compute standardized residuals,

  1. Find the mean of residual, $\bar \epsilon = \frac{\sum_{i=1}^{n} \epsilon_i}{n}$
  2. Calculate the standard deviation of the series, $SD_\epsilon = \sqrt \frac{\sum_{i = 1}^{n} (\epsilon_i - \bar \epsilon)^2}{n} $
  3. Find standardized residual, $s_{\epsilon_i} = \frac{\epsilon_i- \bar \epsilon}{SD_\epsilon}$
Improve this answer
$\endgroup$
2
  • $\begingroup$ My question then is, given the assumptions, is ∑1(yi−y^)^2/ (n−2) equal to ∑(ϵi−ϵ¯)^2/ (n−1) (i.e. the two expressions in my original question) $\endgroup$
    – Mauro Augusto
    Commented Jun 16, 2016 at 9:18
  • $\begingroup$ As @fcop mentioned, provided the assumptions hold, the two are equivalent. $\endgroup$
    – rsl
    Commented Jun 16, 2016 at 9:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.