On page 97 of Introduction to Statistical Learning book, there is a paragraph on studentized residuals within the context of looking for outliers.

But in practice, it can be difficult to decide how large a residual needs to be before we consider the point to be an outlier. To address this problem, instead of plotting the residuals, we can plot the studentized residuals, computed by dividing each residual ei by its estimated standard error.

This gist of this makes sense, I understand that we are looking for residuals that are more than 2-3 deviations away from the mean.

What confused me was use of the word "its"

in each residual ei by its estimated standard error

This implies there is a standard error for each residual ei as opposed to a standard error for all of the residuals as a group. Have I misread or misunderstood this?

When calculating studentized residuals am I dividing each residual by the regression RSE or some other denominator?

up vote 3 down vote accepted

Yes, at least for linear regression, so I will answer for that case. The model is then $$ Y=X\beta+\epsilon $$ in matrix form, where $Y$ is $n\times 1$, $X$ is $n\times p$, $\beta$ is $p\times 1$ and the error term $\epsilon$ is $n\times 1$. We assume the errors $\epsilon_i$ are iid with variance $\sigma^2$ (and expectation zero).

Then the least squares estimator is $\hat{\beta}=(X^T X)^{-1}X^T Y$ and the predicted values is $\hat{Y}= X\hat{\beta}=X (X^TX)^{-1} X Y= HY$ where $H= X (X^TX)^{-1} X$ is the hat matrix. The residual vector $r= Y-\hat{Y}=(I-H)Y$ has expectation zero and variance matrix $\text{Var}(r)= (I-H) \sigma^2 I (I-H) = \sigma^2 (I-H)$ so the standard error of residual component $i$ is $$ \text{se}(r_i) = \sigma \sqrt{1-h_i} $$ and we can show that $ 0\le h_i \le 1$. $h_i$ is often seen as a measure of the influence or leverage of observation $i$, so we can see that the most influential observations have residuals with lesser standard error.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.