7
$\begingroup$

The first graph: residual vs fitted plot

A good residual vs fitted plot has three characteristics:

  • The residuals "bounce randomly" around the 0 line. This suggests that the assumption that the relationship is linear is reasonable.

  • The residuals roughly form a "horizontal band" around the 0 line. This suggests that the variances of the error terms are equal.

  • No one residual "stands out" from the basic random pattern of residuals. This suggests that there are no outliers.

(Source: https://onlinecourses.science.psu.edu/stat501/node/36 link not working January 2019)

In particular, I am no sure what they mean what they mean by a horizontal band in the second point.Is that the red curve in the graph?

So the second point deals with homoscedasticity?

$\endgroup$
4

4 Answers 4

9
$\begingroup$

According to the discussion in Draper and Smith's Applied Regression Analysis (3rd edition, roughly page 59), this residual plot may be used to check for violations in model assumptions particularly related to incorrect specification or presence of heteroscedasticity.

In the case that no violations are detects, the figure might look like the following.

enter image description here

Notice that the residuals are randomly distributed within within the red horizontal lines, forming a horizontal band along the fitted values. There is no visible pattern, which indicates that our regression model specifies an adequate relationship between the outcome, $Y$ and the covariates, $X$.

A figure depicting a potential violation in the model assumptions is

enter image description here

where a horizontal band with a particular width may work well for one part of the data, but might not work so well for another section of the fitted values. In this example, variances for the first quarter of the data, up to about a fitted value of 40 are smaller than variances for fitted values larger than 40. The middle portion of the fitted values has substantially larger variances than the outer values. This indicates that the regression model may have failed to account for heteroscedasticity.

As @ben-bolker mentions in his comments in the linked questions, this diagnostic plot may be even better suited for detection of non-linear relationships that that were not included in the specification. Two reproducible simulated examples of non-linear relationships are presented below. (the R code is presented at the bottom of the post).

The first plot here repeats the ideal scenario, where the regression specification, $Y = \beta_0 + \beta_1 X + \epsilon$, adequately models the underlying relationship. In this instance, the fitted versus residual plot is

enter image description here

where the horizontal red lines are drawn at +- 2. As in the first figure, the points more or less lie in this horizontal band and no residuals are larger than 3 in magnitude (max(abs(regs[[1]]$residuals)) returns 2.932835).

In the second example, the outcome variable has a quadratic relationship with its covariate, $Y = \beta_0 + \beta_1 X + \beta_2 X^2$, but the regression specification only allows for a linear relationship. Here, the fitted versus residual plot shows a fairly strong sign of non-linearity with an upside down "U" shape. This is because the second order term of $X$ has a negative relationship with $Y$.

enter image description here

The third example provides an instance where $\ln Y$ has a linear relationship with X, with $Y = \exp{(\beta_0 + \beta_1 * x + \epsilon)}$ but the model fails account for the needed transformation of $Y$.

enter image description here

Here, the figure indicates a negative trend that isn't accounted for perhaps a bit of a funnel shape indicating heteroscedasticity. Further, there are larger numbers of residuals with extreme values, with 31 out of 500 values larger than 3 and four outside of the plot window, with values of roughly (10.1, 10.5 16.4, and 18.2). This relates to the non-normal error example in @glenn-b's answer to the question linked by @gung above.

data

set.seed(1234)

x <- rnorm(500)
x4 <- (.1 * x) + rnorm(500)
y1 <- 2 * x + rnorm(500)
y2 <- 2 * x + - (.5 * x^2) + rnorm(500)
y3 <- exp(.5 * x + rnorm(500))

# put data into dataframe to organize results
df <- data.frame(x, y1, y2, y3, y4)

# run regressions
regs <- lapply(df[-1], function(y) lm(y ~ x, data=df))
$\endgroup$
6
$\begingroup$

To follow up on @mdewey's answer and disagree mildly with @jjet's: the scale-location plot in the lower left is best for evaluating homo/heteroscedasticity. Two reasons:

  • as raised by @mdewey: it's easier to judge whether the slope of a line than the amount of spread of a point cloud, and easier to fit a nonparametric smooth line to it for visualization purposes
  • a data set with a non-uniform distribution of the fitted value (which is not itself problematic) can fool the viewer into believing there's heteroscedasticity, because your eye tends out to pick out the extremes. Because more observations lead to more extreme residuals (in the sense of order statistics), it will appear that there's more variability in ranges with more data. In this case there are fewer points toward the extremes of the fitted values, which makes it look like the variability is highest in the middle. The scale-location plot avoids this problem.
$\endgroup$
5
$\begingroup$

If you are looking at the top left plot then yes. However the best plot for what you intend is the bottom left one which folds the residuals about the horizontal axis in the first one so that the smoothed line drawn on that plot should be horizontal if there is no relation between scale and location. In your case it looks not too bad as the left hand dip is probably only being driven by a couple of points.

$\endgroup$
3
$\begingroup$

The second point is best evaluated using the top-left plot. Basically, you want to check to see whether the spread of the residuals is the same at all points along the x-axis. If it is, then you'll see a band of points that move horizontally along the x-axis. This would then suggest little evidence of heteroscedasticity. If instead it appears that the points either increase or decrease as you go from right to left, then you might say that "the band of points is increasing/decreasing" rather than staying strictly horizontal. The notion of a "band" of points is really just referring to the overall subjective shape of the scatterplot rather than anything specific.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.