Decomposing $R^2$ into independent variables

Question

Consider a linear regression model: $$y = β_0 + β_1X_1 + β_2X_2 + ... + β_kX_k + ε$$ where $R^2 = 1 - (SSR/SST)$.

I would like to determine the contribution of a factor $i$ (call it $R^2_i$) into the total $R^2$ such that $R^2_1$ + $R^2_2$ + ... +$R^2_k$ = $R^2$, because I want to know the impact of each factor into the total variance.

One methodology, called Shapley-Owen Decomposition, exists that performs that. The problem is that it is very computational heavy because it takes $2^k$ number of computations for each factor. It can be done pretty fast on Python when $k$ is small, but when $k$ is large, it is impossible.

My personal application is to decompose the $R^2$ into factors when $k$ is like 15-20 AND I want to do it using rolling-basis, because I want to see how the contributions change over time (which naturally means more computations).

I am wondering what efficient methodologies are out there to achieve my goal. A reference to academic paper/application would be appreciated. Thank you very much.

Answer 1

I struggled with this same problem--decomposing variance in high-dimensional prediction problems without limiting myself to fitting many, many linear regression models--and came up with the following solution: Shapley Decomposition of R-Squared in Machine Learning Models (with an R implementation).

I would say that the trick in an applied setting is to first find classes of models that support fast Shapley value approximation calculations--e.g., neural networks and gradient boosted decision trees (see the shap Python package)--or use model-agnostic Monte Carlo Shapley value approximations like those implemented in the R package iml: Explaining prediction models and individual predictions with feature contributions. With the trained model, then run the package's predict() or feature importance function to get the resulting n_samples * n_features Shapley value matrix and decompose it according to the first link in my post. And be careful to take note of feature correlation as high correlations can make the feature-level attributions unstable. The authors of the shapr R package have come up with a few interesting approaches to adjusting Shapley values for feature correlations.