22
$\begingroup$

I have just read a paper in which the authors carried out a multiple regression with two predictors. The overall r-squared value was 0.65. They provided a table which split the r-squared between the two predictors. The table looked like this:

            rsquared beta    df pvalue
whole model     0.65   NA  2, 9  0.008
predictor 1     0.38 1.01 1, 10  0.002
predictor 2     0.27 0.65 1, 10  0.030

In this model, ran in R using the mtcars dataset, the overall r-squared value is 0.76.

summary(lm(mpg ~ drat + wt, mtcars))

Call:
lm(formula = mpg ~ drat + wt, data = mtcars)

Residuals:
    Min      1Q  Median      3Q     Max 
-5.4159 -2.0452  0.0136  1.7704  6.7466 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   30.290      7.318   4.139 0.000274 ***
drat           1.442      1.459   0.989 0.330854    
wt            -4.783      0.797  -6.001 1.59e-06 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 3.047 on 29 degrees of freedom
Multiple R-squared:  0.7609,    Adjusted R-squared:  0.7444 
F-statistic: 46.14 on 2 and 29 DF,  p-value: 9.761e-10

How can I split the r-squared value between the two predictor variables?

$\endgroup$
2
  • 1
    $\begingroup$ This post provides information on how to partition the $R^{2}$. $\endgroup$ Jun 4 '13 at 19:08
  • 10
    $\begingroup$ This comment can represent, briefly and inadequately, the point of view that this will often prove futile if not dangerous. The success or failure of a model is best regarded as the result of a team effort by the predictors (and their particular functional forms, interaction terms, etc., etc.) and is to be judged as such. Naturally, most of us are interested in relative importance of predictors and it is not nonsense, but attempts to quantify it exactly need to be accompanied with full statements of the technical and philosophical limitations on such an exercise. $\endgroup$
    – Nick Cox
    Jul 2 '13 at 17:37
9
$\begingroup$

You can just get the two separate correlations and square them or run two separate models and get the R^2. They will only sum up if the predictors are orthogonal.

$\endgroup$
2
  • 2
    $\begingroup$ By 'orthogonal', do you mean the two predictors should be uncorrelated with each other? $\endgroup$
    – luciano
    Jun 4 '13 at 17:30
  • 3
    $\begingroup$ Yes, uncorrelated...it's the only way they sum to the total. $\endgroup$
    – John
    Jun 5 '13 at 4:23
19
$\begingroup$

In addition to John's answer, you may wish to obtain the squared semi-partial correlations for each predictor.

  • Uncorrelated predictors: If the predictors are orthogonal (i.e., uncorrelated), then the squared semi-partial correlations will be the same as the squared zero-order correlations.
  • Correlated predictors: If the predictors are correlated, then the squared semi-partial correlation will represent the unique variance explained by a given predictor. In this case, the sum of squared semi-partial correlations will be less than $R^2$. This remaining explained variance will represent variance explained by more than one variable.

If you are looking for an R function there is spcor() in the ppcor package.

You might also want to consider the broader topic of evaluating variable importance in multiple regression (e.g., see this page about the relaimpo package).

$\endgroup$
0
6
$\begingroup$

I added the tag to your question. Here is part of its tag wiki:

One common method is to add regressors to the model one by one and record the increase in $R^2$ as each regressor is added. Since this value depends on the regressors already in the model, one needs to do this for every possible order in which regressors can enter the model, and then average over orders. This is feasible for small models but becomes computationally prohibitive for large models, since the number of possible orders is $p!$ for $p$ predictors.

Grömping (2007, The American Statistician) gives an overview and pointers to literature in the context of assessing variable importance.

$\endgroup$
4
  • $\begingroup$ Does order actually matter? I mean y ~ a + b is going to be the same as y ~ b + a, isn't it? And yeah, you need to calculate the difference between y ~ a and y ~ a + b as well as y ~ b and y ~ a + b, but you don't actually have to run y ~ b + a, do you? So you'd only actually have to run $2^p$ models (which is viable for a slightly higher p). Please correct me if I'm wrong.. $\endgroup$
    – naught101
    Feb 2 '17 at 4:04
  • $\begingroup$ @naught101: you are right about the order in the model not mattering. However, we are trying to understand the contribution to $R^2$ coming from (say) a. And then, the contribution of a in the absence of b (i.e., the difference in $R^2$ between y~1 and y~a) will usually be quite different than the contribution of 'a' in the presence of b (i.e., the difference in $R^2$ between y~b and y~a+b). So we need to look at all different possible orderings in which 'a' and the other predictors can enter the model. $\endgroup$ Feb 2 '17 at 18:50
  • $\begingroup$ Right, yes, I see. I mis-read the sentence. You need to evaluate $2^p$ models, but also $2!$ model differences. $\endgroup$
    – naught101
    Feb 3 '17 at 0:46
  • $\begingroup$ @naught101: almost correct. There are $2^p=\sum_{q=0}^p{p\choose q}$ models (${p\choose q}$ models containing $q$ out of the $p$ predictors). Except for the trivial model ($q=0$), you want to compare each model with $q$ predictors with another $q$ different submodels, each one of which we arrive at by removing one predictor, so we have $\sum_{q=1}^p q{p\choose q}$ comparisons. (Each model appears multiple times here, and indeed we have more comparisons than $2^p$ models.) And if we have interactions, things become more complicated yet. $\endgroup$ Feb 3 '17 at 10:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.