17
$\begingroup$

Following is a model created from mtcars dataset:

> ols(mpg~wt+am+qsec, mtcars)

Linear Regression Model

ols(formula = mpg ~ wt + am + qsec, data = mtcars)

                Model Likelihood     Discrimination    
                   Ratio Test           Indexes        
Obs       32    LR chi2     60.64    R2       0.850    
sigma 2.4588    d.f.            3    R2 adj   0.834    
d.f.      28    Pr(> chi2) 0.0000    g        6.456    

Residuals

    Min      1Q  Median      3Q     Max 
-3.4811 -1.5555 -0.7257  1.4110  4.6610 

          Coef    S.E.   t     Pr(>|t|)
Intercept  9.6178 6.9596  1.38 0.1779  
wt        -3.9165 0.7112 -5.51 <0.0001 
am         2.9358 1.4109  2.08 0.0467  
qsec       1.2259 0.2887  4.25 0.0002  

The model seems good with total $R^2$ of 0.85. However, partial $R^2$ values seen on following plot do not add up to this value. They add up to approx 0.28.

> plot(anova(mod), what='partial R2')

enter image description here

Is there any relation between sum of all partial $R^2$ and total $R^2$ ? The analysis is done with rms package.

$\endgroup$
1
  • 1
    $\begingroup$ (In addition to the very good answer by amoeba) A close question about standardized regression coefficient vs partial correlation stats.stackexchange.com/q/76815/3277. $\endgroup$
    – ttnphns
    Commented Jun 4, 2015 at 19:48

1 Answer 1

20
$\begingroup$

No.

One way to understand partial $R^2$ for a given predictor is that it equals the $R^2$ that you would get if you first regress your independent variable on all other predictors, take the residuals, and regress those on the remaining predictor.

So if e.g. all predictors are perfectly identical (collinear), one can have decent $R^2$, but partial $R^2$ for all predictors will be exactly zero, because any single predictor has zero additional explanatory power.

On the other hand, if all predictors together explain the dependent variable perfectly, i.e. $R^2=1$, then partial $R^2$ for each predictor will be $1$ too, because whatever is unexplained by all other predictors can be perfectly explained by the remaining one.

So the sum of all partial $R^2$ can easily be below or above the total $R^2$. They do not have to coincide even if all predictors are orthogonal. Partial $R^2$ is a bit of a weird measure.

See this long thread for many more details: Importance of predictors in multiple regression: Partial $R^2$ vs. standardized coefficients.

$\endgroup$
6
  • $\begingroup$ Thanks for a very clear explanation. This could be happening in the situation in this question also: stats.stackexchange.com/questions/155447/… . Then is partial R^2 a reasonable indicator of importance or contribution of individual predictors? Or would you suggest something else like 'proportion R^2' or 'remaining R^2' or 'chisq' or 'chisq minus df' or 'proportion chisq' or 'aic'? All of these are available in rms package. Or standardized coefficients? $\endgroup$
    – rnso
    Commented Jun 4, 2015 at 16:36
  • $\begingroup$ Yes, I am not sure why that question was put on hold as unclear; I think it is clear (and is almost a duplicate of this one but arguably not quite). Regarding reasonable indicators of importance of predictors: I strongly invite you to read the thread I linked to in my answer, which is precisely about this question. There is my own answer there as well, where I make a brief overview of several various indicators. They all have various shortcomings; it seems that there is no (and cannot be a) perfect solution to this problem. $\endgroup$
    – amoeba
    Commented Jun 4, 2015 at 19:16
  • $\begingroup$ I put it on hold as unclear because no explanation at all was given of how the two models differed. Perhaps I was wrong in thinking an answer was wanted for that specific situation. Given this answer, do either of you think it needs re-opening? $\endgroup$
    – Scortchi
    Commented Jun 4, 2015 at 19:31
  • $\begingroup$ So partial R2 are comparable within one graph and not between 2 graphs. Also 'partial R2' does not really indicate the contribution to total R2 and hence I think it is a misnomer. There is no need for other question now since I have got the answer here. $\endgroup$
    – rnso
    Commented Jun 5, 2015 at 1:07
  • $\begingroup$ I have been upvoting all answers. Even then I have accepted vast majority of my questions. $\endgroup$
    – rnso
    Commented Jun 7, 2015 at 1:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.