3
$\begingroup$

A) When considering a simple linear regression model, it is important to check the linearity assumption. Graphing the residuals vs the predictor variable can often give a good idea of whether or not this is true. A non-random pattern suggests that a simple linear model is not appropriate.

B) On the other hand, residuals have the property that the correlation between the residuals and the observations of the predictor variable is zero.

It sounds to me that B) contradicts A). (If the residuals and the predictor are not correlated, how can we see a non-random pattern in the plot?) Can you explain to me why this is not the case?

|cite|improve this question
$\endgroup$
  • $\begingroup$ In an answer at stats.stackexchange.com/questions/152028 I show how to construct such plots where residuals and predictors are uncorrelated but, subject to that constraint, the pattern is practically arbitrary--and can be specified in advance! $\endgroup$ – whuber♦ May 19 '15 at 19:46
4
$\begingroup$

Correlation refers to linear dependence. However, you can have non-linear dependencies. Here is the standard plot from the Wikipedia page on correlation and linear dependence:

enter image description here

The bivariate distributions in the bottom row all have zero correlation, but clear patterns. Thus, although standard (OLS) regression methods enforce a zero correlation between the residuals and the predicted values, there can still be a detectable pattern that indicates the functional form is mis-specified. Consider the following plot, taken from this CV question: How do I interpret this fitted vs residuals plot? As I argue in my answer there, it provides evidence of mis-specified functional form.

enter image description here

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Do I understand correctly that the plot of simple linear regression residuals vs the predictor variable will never look like any in the second row of plots from your wikipedia picture, even if the model if misspecified? (since this would mean that the residuals and the predictor variable can be correlated). $\endgroup$ – user7064 May 19 '15 at 19:37
  • 1
    $\begingroup$ @user7064, they could (in theory) look like the plot in the center of the 2nd row. (They could also look like the plot in the center of the top row, which is more likely.) $\endgroup$ – gung - Reinstate Monica♦ May 19 '15 at 19:40
  • $\begingroup$ Oh yes, of course. But it is impossible to have a plot like any of the six plots with correlation 1 or -1 from the second row. More generally, the plot might exhibit some non-linear association (in which case something has to be done!), but it will never exhibit substantial linear association. Right? $\endgroup$ – user7064 May 19 '15 at 19:43
  • 1
    $\begingroup$ @user7064, it cannot exhibit a linear association unless something has gone wrong, eg if the intercept had been suppressed. $\endgroup$ – gung - Reinstate Monica♦ May 19 '15 at 19:52
  • $\begingroup$ Could you please elaborate a little bit on this? $\endgroup$ – user7064 May 20 '15 at 4:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.