6
$\begingroup$

I'm currently testing the homoscedasticity assumption of multiple linear regression. This can be done by plotting studentized residual plots. The problem with residual plots is even if the variance of the errors is constant, the variance of residuals is not.

My textbook says that using studentized residual-plots instead of residual plots solves this problem. But how?

I think that it is because when fitting the line, we see only the observations Y and not their true value. The fitted line has smaller residual for larger values of x, since larger x-vales have grater leverage. So, the variance of residuals at the end of my fitted line is smaller than in the middle. This leads to heteroscedasticity, even though with "true" Y-values my line would not lead to heteroscedasticity. Studentizing takes (somehow) the leverage of points into account and standardizes them.

Is my explanation correct and why is standardizing the variance needed?

$\endgroup$
1

2 Answers 2

1
$\begingroup$

Your question relates to the variance of the residual vector in multiple linear regression and the resultant studentised residuals (see related answers here and here for some useful discussion on this topic). Under OLS estimation the conditional variance of the (raw) residuals can be shown to be:

$$\mathbb{V}(R_i|\mathbf{x}) = \sigma^2 (1-L_i),$$

where $L_i \equiv [\mathbf{h}]_{i,i}$ is the leverage value for the data point, which is the $i$th diagonal entry of the hat matrix $\mathbf{h}$. We "studentise" the residuals by dividing the raw residuals by their estimated standard deviation:

$$R_{i, \text{Stud}} = \frac{R_i}{\hat{\sigma} \sqrt{1-L_i}}.$$

As in other contexts, the "studentisation" divides the initial quantity by an estimate of its standard deviation, so that it is roughly standardised. This involves estimation of the error variance in the regression, so it is not as perfect as "standardisation" but it is the best we can do since the true error variance is unknown.

It is worth noting that there are actually two main kinds of studentised residuals corresponding to two kinds of estimators used for the error variance. The internally studentised residuals use the standard MSE estimator for the error variance, whereas the externally studentised residuals use an estimator that removes the $i$th data point. Both of these types of residuals roughly adjust for the variance of the raw residuals. (The externally studentised residuals have slightly cleaner statistical properties, since the numerator and denominator are independent; for large samples there is little difference between the two.)

$\endgroup$
0
$\begingroup$

Standardizing the residuals allows you to view them on the standard scale, which can make outlier detection easier. For example, any |residual|>2 implies that your observed value is quite far from your prediction (over 2 standard deviations from your average error). In other words, it helps you to put them on a scale of units (standard deviations) you understand, and it may simplify the detection of extreme cases.

$\endgroup$
1
  • 4
    $\begingroup$ OP mentions studentized not standardized. Studentized are indeed standardized but that is not all that they are. $\endgroup$ Commented Nov 20, 2018 at 13:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.