# Understanding 'predictor' residual plots in multiple regression

Suppose I have two standardized (i.e., z-scored) predictors Age at Marriage and Rate of Marriage, and a dependent variable Divorce and I fit a linear regression model to predict Divorce from the two predictors.

Predictor residual plot

First, we predict Age at Marriage (now as dep. variable) from Rate of Marriage, obtain the residuals from this prediction. Second, we plot these residuals against Divorce .

Picture below is the resultant Predictor residual plot.

Updated Question

This technique can be extended to several-predictor situations. Imagine, for example, we had $3$ predictors predicting $Divorce$. Then, we had to pick one of the predictors, and regress that predictor on the other $2$ predictors, and obtain the residuals and then plot the residual against Divorce.

My question is what would be the interpretation of such plots in 3-or more predictor situations?

When you regress Age of marriage on Rate of marriage, each of your residuals is computed as:

residual = Age of marriage - (Estimated) Expected Age of marriage

where (Estimated) Expected Age of marriage = b0 + b1xRate of marriage" and b0 and b1 are the estimated intercept and slope of the simple linear regression of *Age of marriage" on Rate of marriage.

When is a residual positive? When:

Age of marriage - (Estimated) Expected Age of marriage > 0

or, equivalently, when

Age of marriage > (Estimated) Expected Age of marriage. In other words, when the *Age of marriage" is older than the (estimated) expected *Age of marriage".

When is a residual negative? When:

Age of marriage - (Estimated) Expected Age of marriage < 0

or, equivalently, when

Age of marriage < (Estimated) Expected Age of marriage. In other words, when the *Age of marriage" is younger than the (estimated) expected *Age of marriage".

To answer the updated question, the interpretation of when a residual is positive or negative would still be the same as per my original answer. However, given the predictors, the (Estimated) Expected Age of marriage involved in the definition of the residuals would now be defined differently, namely:

*(Estimated) Expected Age of marriage" = b0 + b1*Rate of marriage + b2*Predictor2 + b3*Predictor3"

if we had two additional predictors, etc. So interpretations will be conditional on three predictors in this example, rather than just one predictor.

• We finally plot the residuals against Divorce (our dependent var.) but I don't see any mention of Divorce in your interpretations? Let's be more concrete. Suppose we have another predictor (no. 3) called self-confidence. So, (Estimated) Expected Age of marriage (*itself a predictor!)" = b0 + b1*Rate of marriage + b2*Self-confidence*, and we get the residuals. We then plot these residuals against Divorce. Can you please clearly explain how we interpret this new Predictor Residual Plot in terms datapoints to the left and to the right of the Vertical Dashed Line? – rnorouzian Mar 20 '18 at 3:36
• Normally, we prefer additional information to an old answe be given as an update to the old answer rather than as a new answer. – kjetil b halvorsen Apr 18 '18 at 17:33

The residuals from any regression model are expressed as the difference between the actual response and the (estimated) expected value of the response. If your model regresses Age of marriage on Rate of marriage and Self-confidence, the residuals corresponding to that model represent the difference between the actual Age of marriage and its (estimated) expected value. Those residuals are positive when this difference is positive, hence when Age of marriage is older than the (estimated) expected Age of mariage, etc.

When you plot Divorce rate against these residuals and draw that vertical line through zero, you are essentially asking when the residuals are positive or negative. Just as explained above, they are positive when the difference between the actual Age of marriage and its (estimated) expected value from the model regressing Age of marriage on Rate of marriage and Self-confidence is positive. They are negative when this difference is negative. Whether they are positive or negative has nothing to do with Divorce rate - rather, it is influenced by Rate of marriage and Self-confidence.

Unless you are interested in fitting different linear relationships between Divorce rate and Age of marriage residuals for positive and negative residuals, I think you should interpret the plot of Divorce rate versus Age of marriage residuals as capturing the linear relationship between Divorce rate and whatever is left over after removing the linear effects of Rate of marriage and Self-confidence from Age of marriage. Why you would want to do something like this it's not clear to me. It sure complicates the interpretation of your final model.

If this is not clear enough, perhaps others on this forum might be able to clarify it.

• Aha!! Why you would want to do something like this it's not clear to me. It sure complicates the interpretation of your final model. This was what I wanted to hear ! So, when we have 3 or more predictors, the Predictor Residual Plot is not so helpful, so what is more helpful in that case then? – rnorouzian Mar 20 '18 at 4:28
• @Isabella The standard method on this site is to edit your original answer to address additional questions. – Matthew Drury Mar 20 '18 at 4:55
• @MatthewDrury, Isabella so kindly answered any question that came up, and so generously shared her knowledge with me. She is new to our community. Would you mind upvoting her posts to support and encourage her to keep on doing so? -- Thank you very much in advance. – rnorouzian Mar 20 '18 at 5:15
• @MatthewDrury, thank you for letting me know what the standard method is. I'll keep that in mind for subsequent answers. I am indeed new to the site and have experienced having answers deleted, etc., which is not necessarily as user-friendly as I would have expected, but I'll try to persevere while I learn my way around. – Isabella Ghement Mar 20 '18 at 16:02
• I upvoted isabella's first answer. – Matthew Drury Mar 20 '18 at 16:12