Having read about studentized residuals I do not understand how the idea of different residual variances conditional on certain values of a predictor $X$ (as implied by the concept of studentized residuals) is not inherently conflicting with the assumption of homoscedasticity in linear regression models with a single predictor variable.

It says in my textbook that the assumption of homoscedasticity means that the variance of $Y$ (dependent variable) conditional on $X = x$ (a certain realization of the independent predictor variable) is constant across the value range of this predictor. This conditional variance, it says, is also equal to the conditional variance of the residual variable $\varepsilon$ for a given $x$. To my understanding this is a statement at the population level. Together that would be:

$Var(Y|X) = Var(\varepsilon|X) = \sigma^2_\varepsilon$

Later the book deals with the detection of outliers in the dependent variable and suggests using standardized and studentized residuals. A standardized residual is an individual residual $\varepsilon_i$ divided by the estimated standard deviation $\hat\sigma_\varepsilon$ of the residual variable in the population. In the case of the standardized residual each residual $\varepsilon_i$ is thus standardized using the same constant value $\hat\sigma_\varepsilon$ if homoscedasticity can be assumed:

$Stand.Res_i = \frac{\varepsilon_i}{\hat\sigma_\varepsilon}$.

However, in the next paragraph the studentized residual is introduced. The book says: "It can be shown that the precision of the estimation of the residuals increases with the distance of $x_i$ from its mean $\bar x$. In the case of the studentized residual, residuals are not divided by their overall estimated standard error but by the estimated standard deviation of the residuals at the location $x_i$. This standard deviation can be obtained from this formula:

$Student.Res_i = \frac{\varepsilon_i }{\hat\sigma_\varepsilon \cdot \sqrt {1-h_i}} $

with $h_i$ being the leverage score of a (in this simple case: singular) predictor $x_i$. So it seems to me that in this case the residuals are not all divided by the same constant value (like they were in the case of the standardized residual) but instead there is now a distribution of residual standard errors that depends on the leverage values. These leverage values are larger at the extreme ends of the predictor variable as has been explained in other questions on this site. On Wikipedia (https://en.wikipedia.org/wiki/Errors_and_residuals#Regressions) it says:

In regression analysis, the distinction between errors and residuals is subtle and important, and leads to the concept of studentized residuals. Given an unobservable function that relates the independent variable to the dependent variable – say, a line – the deviations of the dependent variable observations from this function are the unobservable errors. If one runs a regression on some data, then the deviations of the dependent variable observations from the fitted function are the residuals. [...] However, because of the behavior of the process of regression, the distributions of residuals at different data points (of the input variable) may vary even if the errors themselves are identically distributed. Concretely, in a linear regression where the errors are identically distributed, the variability of residuals of inputs in the middle of the domain will be higher than the variability of residuals at the ends of the domain[citation needed]: linear regressions fit endpoints better than the middle. This is also reflected in the influence functions of various data points on the regression coefficients: endpoints have more influence.

While this makes sense to me intuitively I don't quite understand how it doesn't contradict the assumption of homoscedasticity. Is this because at the population level the error variance can be equal for all levels of $X$ but when we estimate that error variance by fitting a regression line (so that we can then use the residuals as estimates of the errors at population level) we automatically and artificially create a distribution of residual standard deviations conditional on $X$ instead of having that residual standard deviation be the equal singular value for every residual? So that would mean that the standardized residual is only really useful at the (unobservable) population level, right? Because for a given sample the standardized residual can impossibly be an accurate estimator for all values $x_i$ that are far from $\bar x$ simply due to the way that the regression model is fitted?

However, if that were the case I don't understand the recommendation that I have read in so many places to test for homoscedasticity by plotting the studentized residuals against the predictor variable and testing if the variance of the residuals will be equal at all levels of $X$. If in a fitted regression line the variance of residuals can be different for different levels of $X$ (as I think the concept of the studentized residual and that excerpt from Wikipedia imply) how does it make sense that everybody recommends testing the equality of residual variances across $X$ by using the scatterplots of studentized residuals against the level of the predictor $X$? Can somebody please point out what mistakes I am making here?

Do I mix up error and residual values or population and sample levels? I have been looking everywhere and couldn't find a satisfying answer that discussed why this is not a conflict (or at least none that I understood). Thank you so much in advance!


1 Answer 1


I do not really understand your confusion, but let me give this a try. Consider a linear regression $$ y=X\beta+\varepsilon $$ with errors $\varepsilon$ and residuals $e:=y-X\hat\beta=(I-H)y$ where $I$ is an identity matrix and $H:=X(X^\top X)^{-1}X^\top$ is the hat matrix. Suppose the linear model is correctly specified and all the assumptions, including unconditional and conditional homoskedasticity of errors, are met.

While $\varepsilon$ are homoskedastic by the assumption I just introduced, the model residuals $e$ are conditionally heteroskedastic w.r.t the level of $X$: their variance can be shown to be $\text{Var}(e)=\sigma^2_\varepsilon(I-H)$. This is an artifact of OLS estimation in a linear model.

Now suppose you do not know whether all of the assumptions are met (which is the realistic perspective) and you would like to check them. You would perhaps be tempted to use the residuals $e$ in place of the unobserved errors $\varepsilon$ to do model diagnostics, e.g. assess the assumption of conditional homoskedasticity of $\varepsilon$. Unfortunately, a conditionally homoskedastic $\varepsilon$ translates into a conditionally heteroskedastic $e$ as evidenced by the variance formula above. Thus you cannot learn much about the conditional homoskedasticity of $\varepsilon$ by inspecting the variability in $e$ vs. $X$.

But there is a remedy. You can adjust for the variance distortion in $e$ by "undoing" the scaling due to multiplication by $(I-H)$ in $e$. This results in (internally or externally) studentized residuals $\tilde{e}_{int}:=\frac{e}{\hat\sigma_{int}\sqrt{1-h_{ii}}}$ or $\tilde{e}_{ext}:=\frac{e}{\hat\sigma_{ext}\sqrt{1-h_{ii}}}$ where $\hat\sigma_{int}$ and $\hat\sigma_{ext}$ are internal and external estimates of error variance, respectively. Studentization of residuals allows putting the residuals back to the same level of conditional variance as the unobserved model errors $\varepsilon$ are, up to a scaling factor that is uniform across the data points and thus does not affect conditional homo- or heteroskedasticity.

This is why it makes sense to use studentized residuals $\tilde{e}$ in place of raw residuals $e$ when assessing conditional heteroskedasticity of the model errors $\varepsilon$ w.r.t. to the regressor $X$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.