10
$\begingroup$

The reason I ask this is because it seems that internally studentized residuals seem to have the same pattern as raw estimated residuals. It would be great if someone could offer an explanation.

$\endgroup$
13
$\begingroup$

Assume a regression model $\bf{y} = \bf{X} \bf{\beta} + \bf{\epsilon}$ with design matrix $\bf{X}$ (a $\bf{1}$ column followed by your predictors), predictions $\hat{\bf{y}} = \bf{X} (\bf{X}' \bf{X})^{-1} \bf{X}' \bf{y} = \bf{H} \bf{y}$ (where $\bf{H}$ is the "hat-matrix"), and residuals $\bf{e} = \bf{y} - \hat{\bf{y}}$. The regression model assumes that the true errors $\bf{\epsilon}$ all have the same variance (homoskedasticity):

homoskedasticity

The covariance matrix of the residuals is $V(\bf{e}) = \sigma^{2} (\bf{I} - \bf{H})$. This means that the raw residuals $e_{i}$ have different variances $\sigma^{2} (1-h_{ii})$ - the diagonal of the matrix $\sigma^{2} (\bf{I} - \bf{H})$. The diagonal elements of $\bf{H}$ are the hat-values $h_{ii}$.

The truely standardized residuals with variance 1 throughout are thus $\bf{e} / (\sigma \sqrt{1 - h_{ii}})$. The problem is that the error variance $\sigma$ is unknown, and internally / externally studentized residuals $\bf{e} / (\hat{\sigma} \sqrt{1 - h_{ii}})$ result from particular choices for an estimate $\hat{\sigma}$.

Since raw residuals are expected to be heteroskedastic even if the $\epsilon$ are homoskedastic, the raw residuals are theoretically less well suited to diagnose problems with the homoskedasticity assumption than standardized or studentized residuals.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ The definitional differences between the two different types of residuals (as well as externally studentized residuals) are clear to me. In practice, however, I don't think I have encountered cases (at least with my own data) where internally studentized residuals have a distinct pattern in comparison to estimated residuals. On the other hand, externally studentized residuals can potentially exhibit a distinct pattern than estimated residuals. *I am not saying that the two types of residuals are the same. I am referring to their general patterns. $\endgroup$ – user11392 Nov 20 '12 at 17:24
  • $\begingroup$ @AlexH. I agree that the stated advantage I lated added is theoretical. Constructing a simulated empirical situation where raw residuals are misleading, and studentized residuals provide a more accurate picture of the conditional distributions would be a good addition. $\endgroup$ – caracal Nov 20 '12 at 18:27
12
$\begingroup$

What types of data did you do your test plots on? When all the assumptions hold (or come close) then I would not expect much of a difference between the raw and studentized residuals, the main advantage is when there are highly influential points. Consider this (simulated) data that has a positive linear trend and a highly influential outlier:

enter image description here

Here is the plot of the fitted values vs. the raw residuals:

enter image description here

Notice that the value of the residual of our influential point is closer to 0 than the minimum and maximum residuals from the rest of the points (it is not in the 3 most extreme raw residuals).

Now here is the plot with the standardized (internally studentized) residuals:

enter image description here

In this plot the standardized residual stands out because its influence has been accounted for.

In this simple example it is easy to see what is going on, but what if we had more than 1 $x$ variable and a point that was very influential, but not unusual in the 2 dimensional plots? It would not be obvious from plots of raw residuals, but the studentized residuals would show that residual as more extreme.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy