35
$\begingroup$

I have a model of Movies dataset and I used the regression:

model <- lm(imdbVotes ~ imdbRating + tomatoRating + tomatoUserReviews+ I(genre1 ** 3.0) +I(genre2 ** 2.0)+I(genre3 ** 1.0), data = movies)
library(ggplot2)
res <- qplot(fitted(model), resid(model))
res+geom_hline(yintercept=0)

Which gave the output:

enter image description here

Now I tried working something called Added Variable Plot first time and I got the following output:

car::avPlots(model, id.n=2, id.cex=0.7)

Added Variable Plot

The problem is I tried to understand Added Variable Plot using google but I couldn't understand its depth, seeing the plot I understood that its kind of representation of skewing based on each of the input variable related to the output.

Can I get bit more details like how its justifies the data normalization?

$\endgroup$
5
  • 5
    $\begingroup$ @Silverfish has given a nice answer to your question. On the small detail of what to do with your particular dataset, a linear model looks like a very bad idea. Votes is manifestly a highly skewed non-negative variable, so something like a Poisson model is indicated. See e.g. blog.stata.com/tag/poisson-regression Note that such a model doesn't commit you to the assumption that the marginal distribution of the response is exactly Poisson any more than a standard linear model commits you to postulating marginal normality. $\endgroup$
    – Nick Cox
    Commented Nov 26, 2014 at 14:25
  • 2
    $\begingroup$ One way of seeing that the linear model works poorly is to note that it predicts negative values for a substantial fraction of cases. See the region left of fitted $ = 0$ on the first residual plot. $\endgroup$
    – Nick Cox
    Commented Nov 26, 2014 at 14:40
  • $\begingroup$ Thanks Nick Cox, here I found that there is a highly skewed non-negative nature , I must consider Poisson model , so is there any link which gives me a proper idea about which model to use in which scenario based on dataset and I tried using Polynomial regression for my dataset , will that be a right choice here ... $\endgroup$ Commented Nov 26, 2014 at 14:53
  • 1
    $\begingroup$ I've already given a link which in turn gives further references. Sorry, but I don't understand the second half of your question with reference to "scenario based on dataset" and "polynomial regression". I suspect you need to ask a new question with much more detail. $\endgroup$
    – Nick Cox
    Commented Nov 26, 2014 at 15:18
  • $\begingroup$ What package did you install so that R recognize the function avPlots ? $\endgroup$
    – user208618
    Commented Dec 9, 2018 at 6:18

2 Answers 2

52
$\begingroup$

For illustration I will take a less complex regression model $Y = \beta_1 + \beta_2 X_2 + \beta_3 X_3 + \epsilon$ where the predictor variables $X_2$ and $X_3$ may be correlated. Let's say the slopes $\beta_2$ and $\beta_3$ are both positive so we can say that (i) $Y$ increases as $X_2$ increases, if $X_3$ is held constant, since $\beta_2$ is positive; (ii) $Y$ increases as $X_3$ increases, if $X_2$ is held constant, since $\beta_3$ is positive.

Note that it's important to interpret multiple regression coefficients by considering what happens when the other variables are held constant ("ceteris paribus"). Suppose I just regressed $Y$ against $X_2$ with a model $Y = \beta_1' + \beta_2' X_2 + \epsilon'$. My estimate for the slope coefficient $\beta_2'$, which measures the effect on $Y$ of a one unit increase in $X_2$ without holding $X_3$ constant, may be different from my estimate of $\beta_2$ from the multiple regression - that also measures the effect on $Y$ of a one unit increase in $X_2$, but it does hold $X_3$ constant. The problem with my estimate $\hat{\beta_2'}$ is that it suffers from omitted-variable bias if $X_2$ and $X_3$ are correlated.

To understand why, imagine $X_2$ and $X_3$ are negatively correlated. Now when I increase $X_2$ by one unit, I know the mean value of $Y$ should increase since $\beta_2 > 0$. But as $X_2$ increases, if we don't hold $X_3$ constant then $X_3$ tends to decrease, and since $\beta_3 > 0$ this will tend to reduce the mean value of $Y$. So the overall effect of a one unit increase in $X_2$ will appear lower if I allow $X_3$ to vary also, hence $\beta_2' < \beta_2$. Things get worse the more strongly $X_2$ and $X_3$ are correlated, and the larger the effect of $X_3$ through $\beta_3$ - in a really severe case we may even find $\beta_2' < 0$ even though we know that, ceteris paribus, $X_2$ has a positive influence on $Y$!

Hopefully you can now see why drawing a graph of $Y$ against $X_2$ would be a poor way to visualise the relationship between $Y$ and $X_2$ in your model. In my example, your eye would be drawn to a line of best fit with slope $\hat{\beta_2'}$ that doesn't reflect the $\hat{\beta_2}$ from your regression model. In the worst case, your model may predict that $Y$ increases as $X_2$ increases (with other variables held constant) and yet the points on the graph suggest $Y$ decreases as $X_2$ increases.

The problem is that in the simple graph of $Y$ against $X_2$, the other variables aren't held constant. This is the crucial insight into the benefit of an added variable plot (also called a partial regression plot) - it uses the Frisch-Waugh-Lovell theorem to "partial out" the effect of other predictors. The horizontal and vertical axes on the plot are perhaps most easily understood* as "$X_2$ after other predictors are accounted for" and "$Y$ after other predictors are accounted for". You can now look at the relationship between $Y$ and $X_2$ once all other predictors have been accounted for. So for example, the slope you can see in each plot now reflects the partial regression coefficients from your original multiple regression model.

A lot of the value of an added variable plot comes at the regression diagnostic stage, especially since the residuals in the added variable plot are precisely the residuals from the original multiple regression. This means outliers and heteroskedasticity can be identified in a similar way to when looking at the plot of a simple rather than multiple regression model. Influential points can also be seen - this is useful in multiple regression since some influential points are not obvious in the original data before you take the other variables into account. In my example, a moderately large $X_2$ value may not look out of place in the table of data, but if the $X_3$ value is large as well despite $X_2$ and $X_3$ being negatively correlated then the combination is rare. "Accounting for other predictors", that $X_2$ value is unusually large and will stick out more prominently on your added variable plot.

$*$ More technically they would be the residuals from running two other multiple regressions: the residuals from regressing $Y$ against all predictors other than $X_2$ go on the vertical axis, while the residuals from regression $X_2$ against all other predictors go on the horizontal axis. This is really what the legends of "$Y$ given others" and "$X_2$ given others" are telling you. Since the mean residual from both of these regressions is zero, the mean point of ($X_2$ given others, $Y$ given others) will just be (0, 0) which explains why the regression line in the added variable plot always goes through the origin. But I often find that mentioning the axes are just residuals from other regressions confuses people (unsurprising perhaps since we now are talking about four different regressions!) so I have tried not to dwell on the matter. Comprehend them as "$X_2$ given others" and "$Y$ given others" and you should be fine.

$\endgroup$
10
  • 1
    $\begingroup$ Not sure how to ask this, but is there anything that can really be said about the trends seen in the plots? For example does the goodness of fit of each trend relate to how independent each of the predictors are, or something like that? $\endgroup$
    – naught101
    Commented Aug 17, 2016 at 5:33
  • 3
    $\begingroup$ Does a method exist for translating the units of residual on the horizontal and vertical axes into units of the underlying variables? $\endgroup$
    – Nicholas G
    Commented Nov 5, 2016 at 12:54
  • $\begingroup$ This is such an excellent answer. But is there a typo in your first paragraph (predictor variables)? Should they be X2 and X3? $\endgroup$
    – detly
    Commented Jul 25, 2019 at 5:19
  • 1
    $\begingroup$ @Arthur There are 4 different regressions as I mention at the end. There's the original regression. There are the two regressions (you mention one of them but the same happens on the x axis) whose residuals produce the new x and new y. And then there's the new regression with the new x and new y. And the great joy of the Frisch–Waugh–Lovell theorem is that the slope, standard error and residuals of this new regression are the same as the original regression! For a reference, try Dougherty's Introduction to Econometrics, OUP. $\endgroup$
    – Silverfish
    Commented Jun 28 at 13:33
  • 1
    $\begingroup$ @BigBen Fixed thanks! To avoid comment clutter I'll delete this comment tomorrow. Feel free to delete yours too. $\endgroup$
    – Silverfish
    Commented Dec 11 at 17:37
-1
$\begingroup$

is there anything that can really be said about the trends seen in the plots

Sure, their slopes are the regression coefficients from the original model (partial regression coefficients, all other predictors held constant)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.