We know that in simple linear regression the variance of the regression error, $\sigma^2$, is estimated by $\frac {\sum_{i=1}^{n} (y_i - \hat y)^2} {n-2}$, i.e., the Mean Squared Error of the errors. But to standardize the residuals it is said to use the "standard error" of the residuals. Is this the exact same thing as that formula I just wrote?

Or is it $\frac{\sum_{i = 1}^{n} (\epsilon_i - \bar \epsilon)^2}{n-1}$, assuming $\epsilon_i$ is the $i$th residual?. This makes more sense to me. Are the two somehow equivalent?

  • 1
    $\begingroup$ If all assumptions are fulfilled then $\bar{e}=0$ and as $y_i -\hat{y}_i=e_i$ they are equivalent $\endgroup$
    – user83346
    Jun 16 '16 at 5:23
  • $\begingroup$ @Wolfgang, to calculate $\sigma^2$, why we divide by $n-2$ instead of $n-1$? $\endgroup$
    – rsl
    Jun 16 '16 at 7:05
  • 1
    $\begingroup$ @Moazzem Hossen We divide by n-2 to get an unbias estimate of the variance $\endgroup$ Jun 16 '16 at 9:20
  • $\begingroup$ Do you use Weighted least squares? In thic case It's suppused that errors depend of regressors and have different variances. $\endgroup$
    – mmmmmmm
    Dec 30 '16 at 10:30
  • $\begingroup$ The term standard error has nothing to do with standardizing residuals. $\endgroup$
    – Nick Cox
    Dec 30 '16 at 10:54

The standardized residual is the residual divided by its standard deviation. Standardizing residual is a method for transforming data so that its mean is zero and standard deviation is one. If the distribution of the residuals is approximately normal, then $95\%$ of the standardized residuals should fall between $-2$ and $+2$. If many of the residuals fall outside of $+2$ or $–2$, then they could be considered unusual. However, about $5\%$ of the residuals could fall outside of this region due to chance. Consider the regression equation $\hat y_i = \beta_0 + \beta_1x_i + \epsilon_i$ and to compute standardized residuals,

  1. Find the mean of residual, $\bar \epsilon = \frac{\sum_{i=1}^{n} \epsilon_i}{n}$
  2. Calculate the standard deviation of the series, $SD_\epsilon = \sqrt \frac{\sum_{i = 1}^{n} (\epsilon_i - \bar \epsilon)^2}{n} $
  3. Find standardized residual, $s_{\epsilon_i} = \frac{\epsilon_i- \bar \epsilon}{SD_\epsilon}$
  • $\begingroup$ My question then is, given the assumptions, is ∑1(yi−y^)^2/ (n−2) equal to ∑(ϵi−ϵ¯)^2/ (n−1) (i.e. the two expressions in my original question) $\endgroup$ Jun 16 '16 at 9:18
  • $\begingroup$ As @fcop mentioned, provided the assumptions hold, the two are equivalent. $\endgroup$
    – rsl
    Jun 16 '16 at 9:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.