How to determine relative contribution of explanatory variables in a linear regression

Question

I estimate a linear regression model, for instance,

$$Y = \alpha + \beta_1 X_1 + \beta_2 X_2 + u$$

and I want to determine how much the variables contribute to $Y$ on average. Put in other words, I want to decompose the explained variance in $Y$ in the contributions of the two explanatory variables.

I was thinking to multiply the estimated $\hat{\beta}$s with the average of the $X$s. This would allow me, for instance, to conclude that $Y$ is on average 20, the constant is 5 and on average $X_1$ contributes by 12 and $X_2$ by 3 to $Y$. But while googling I also came across methods such as Shaply value approach or Relative Weight Analysis (RWA).

Is there a standard way to do such a decompositon (in econometrics)?

Answer 1

If $\mathcal{M}_1$ is your full model and you want to know the contribution of $X_1$ for example, then you fit another model $\mathcal{M}_2$ that includes everything but $X_1$. You can then compute a partial $R^2$ using both models' sums of squared errors ($SS$) as follows:

$$R^2_\mathrm{partial} = \frac{SS_{\mathcal{M}_2} - SS_{\mathcal{M}_1}}{SS_{\mathcal{M}_2}}$$

In R, there are packages that can do this for you, like sensemakr:

library("sensemakr")
LM <- lm(Sepal.Length ~ Sepal.Width + Petal.Length, data = iris)

partial_r2(LM)
# (Intercept)  Sepal.Width Petal.Length 
#   0.3588317    0.3342001    0.8379377 

From the help file:

The partial R2 describes how much of the residual variance of the outcome (after partialing out the other covariates) a covariate explains.

Answer 2

Shapley values are a good alternative to the partial $R^2$ recommended by Frans Rodenburg.

Shapley values, for example as implemented by the package relaimpo::calc.relimp() or variants in dominance analysis such as those computed by parameters::dominance_analysis() offer a systematic way to ascribe the overlap between predictive variables by averaging over all the ways in which the variable could be included, sequentially, in the model.

The Shapley values computed by relaimpo::calc.relimp() show similar results as the partial $R^2$ in that it flags Petal.Length as having a bigger 'effect' on Sepal.Length.

> lm(Sepal.Length ~ Sepal.Width + Petal.Length, data = iris) |> 
   relaimpo::calc.relimp()

Response variable: Sepal.Length 
Total response variance: 0.6856935 
Analysis based on 150 observations 

2 Regressors: 
Sepal.Width Petal.Length 
Proportion of variance explained by model: 84.02%
Metrics are not normalized (rela=FALSE). 

Relative importance metrics: 

                    lmg
Sepal.Width  0.04702292
Petal.Length 0.79315491

Average coefficients for different model sizes: 

                     1X       2Xs
Sepal.Width  -0.2233611 0.5955247
Petal.Length  0.4089223 0.4719200

parameters::dominance_analysis() provides the same results with a bit more detail in the Conditional Dominance Statistics results that shows the $\Delta R^2$ obtained when included either first or second for both predictive variables. These values are averaged to get the Shapley value/General Dominance Statistics results.

> lm(Sepal.Length ~ Sepal.Width + Petal.Length, data = iris) |> 
   parameters::dominance_analysis()

# Dominance Analysis Results

Model R2 Value:  0.840 

General Dominance Statistics

Parameter    | General Dominance | Percent | Ranks |       Subset
-----------------------------------------------------------------
(Intercept)  |                   |         |       |     constant
Sepal.Width  |             0.047 |   0.056 |     2 |  Sepal.Width
Petal.Length |             0.793 |   0.944 |     1 | Petal.Length

Conditional Dominance Statistics

Subset       | IVs: 1 | IVs: 2
------------------------------
Sepal.Width  |  0.014 |  0.080
Petal.Length |  0.760 |  0.826

Complete Dominance Designations

Subset       | < Sepal.Width | < Petal.Length
---------------------------------------------
Sepal.Width  |               |           TRUE
Petal.Length |         FALSE |               

It is worth noting also that the final column of the Conditional Dominance Statistics for both predictive variables are the semipartial (not partial) $R^2$ values between the predictive variable and the outcome controlling for the other predictive factors.