Interpreting plot.lm()

Question

I had a question about interpreting the graphs generated by plot(lm) in R. I was wondering if you guys could tell me how to interpret the scale-location and leverage-residual plots? Any comments would be appreciated. Assume basic knowledge of statistics, regression and econometrics.

Answer 1

As stated in the documentation, plot.lm() can return 6 different plots:

[1] a plot of residuals against fitted values, [2] a Scale-Location plot of sqrt(| residuals |) against fitted values, [3] a Normal Q-Q plot, [4] a plot of Cook's distances versus row labels, [5] a plot of residuals against leverages, and [6] a plot of Cook's distances against leverage/(1-leverage). By default, the first three and 5 are provided. (my numbering)

Plots [1], [2], [3] & [5] are returned by default. Interpreting [1] is discussed on CV here: Interpreting residuals vs. fitted plot for verifying the assumptions of a linear model. I explained the assumption of homoscedasticity and the plots that can help you assess it (including scale-location plots [2]) on CV here: What does having constant variance in a linear regression model mean? I have discussed qq-plots [3] on CV here: QQ plot does not match histogram and here: PP-plots vs. QQ-plots. There is also a very good overview here: How to interpret a QQ-plot? So, what's left is primarily just understanding [5], the residual-leverage plot.

To understand this, we need to understand three things:

• leverage,
• standardized residuals, and
• Cook's distance.

To understand leverage, recognize that Ordinary Least Squares regression fits a line that will pass through the center of your data, $(\bar X,~\bar Y)$. The line can be shallowly or steeply sloped, but it will pivot around that point like a lever on a fulcrum. We can take this analogy fairly literally: because OLS seeks to minimize the vertical distances between the data and the line*, the data points that are further out towards the extremes of $X$ will push / pull harder on the lever (i.e., the regression line); they have more leverage. One result of this could be that the results you get are driven by a few data points; that's what this plot is intended to help you determine.

Another result of the fact that points further out on $X$ have more leverage is that they tend to be closer to the regression line (or more accurately: the regression line is fit so as to be closer to them) than points that are near $\bar X$. In other words, the residual standard deviation can differ at different points on $X$ (even if the error standard deviation is constant). To correct for this, residuals are often standardized so that they have constant variance (assuming the underlying data generating process is homoscedastic, of course).

One way to think about whether or not the results you have were driven by a given data point is to calculate how far the predicted values for your data would move if your model were fit without the data point in question. This calculated total distance is called Cook's distance. Fortunately, you don't have to rerun your regression model $N$ times to find out how far the predicted values will move, Cook's D is a function of the leverage and standardized residual associated with each data point.

With these facts in mind, consider the plots associated with four different situations:

1. a dataset where everything is fine
2. a dataset with a high-leverage, but low-standardized residual point
3. a dataset with a low-leverage, but high-standardized residual point
4. a dataset with a high-leverage, high-standardized residual point

The plots on the left show the data, the center of the data $(\bar X,~\bar Y)$ with a blue dot, the underlying data generating process with a dashed gray line, the model fit with a blue line, and the special point with a red dot. On the right are the corresponding residual-leverage plots; the special point is 21. The model is badly distorted primarily in the fourth case where there is a point with high leverage and a large (negative) standardized residual. For reference, here are the values associated with the special points:

                              leverage std.residual   cooks.d
high leverage,  low residual 0.3814234    0.0014559 0.0000007
low leverage,  high residual 0.0476191    3.4456341 0.2968102
high leverage, high residual 0.3814234   -3.8086475 4.4722437

Below is the code I used to generate these plots:

set.seed(20)

x1 = rnorm(20, mean=20, sd=3)
y1 = 5 + .5*x1 + rnorm(20)

x2 = c(x1, 30);        y2 = c(y1, 20.8)
x3 = c(x1, 19.44);     y3 = c(y1, 20.8)
x4 = c(x1, 30);        y4 = c(y1, 10)

* _{For help understanding how OLS regression seeks to find the line that minimizes the vertical distances between the data and the line, see my answer here: What is the difference between linear regression on y with x and x with y?}