21
votes
Comparison between SHAP (Shapley Additive Explanation) and LIME (Local Interpretable Model-Agnostic Explanations)
LIME creates a surrogate model locally around the unit whose prediction you wish to understand. Thus it is inherently local.
Shapley values 'decompose' the final prediction into the contribution of ...
- 211
12
votes
Accepted
Can I scale and then interpret shap values as percent contribution to the prediction?
Almost yes. There are a few caveats regarding directly interpreting the scaled SHAP values, as the percentage contributions of our final classification prediction for a single observation.
Raw SHAP ...
- 46k
12
votes
Accepted
If feature importance is only computed based on training set, does it mean one should never compute shap values on test set?
Contrary to standard feature importance, calculating SHAP values on a hold-out set does not make a big difference because ultimately we expect our model to have similar behaviour for instances in the ...
- 46k
11
votes
Accepted
Why can a model's SHAP values change on a new dataset?
Why can a model's SHAP values change on a new dataset?
WHY SHOULDN'T THEY?
When you calculate a mean on a new data set, you expect to get a slightly (or radically) different value.
When you calculate ...
- 67.2k
8
votes
Accepted
Are SHAP values potentially misleading when predictors are highly correlated?
Summary
Yes, SHAP values are potentially misleading when predictors are correlated -- they can be imprecise and even have the opposite sign.
The correlation does not need to be incredibly high, around ...
- 373
7
votes
Accepted
What is the use of expected value in machine learning models?
It is the simplest (or amongst the simplest) possible model(s). If your explanatory data (feature list) is $\mathcal D$ and the target variable is $Y$, what models usually try is predicting the ...
- 58.2k
7
votes
What is Shapley value regression and how does one implement it?
The Shapley Value Regression: Shapley value regression significantly ameliorates the deleterious effects of collinearity on the estimated parameters of a regression equation. The concept of Shapley ...
- 71
6
votes
Is there any reason to use LIME now that shap is available?
I wouldn't say that LIME is a flawed half-solution and that SHAP is a perfect full solution.
If anything, I would say both solutions are inherently flawed but perhaps are the best we have. If you are ...
- 528
6
votes
Are SHAP values potentially misleading when predictors are highly correlated?
By default, TreeExplainer in the SHAP (SHapley Additive exPlanations) library uses feature_perturbation = "interventional". This choice is based on the ...
5
votes
Accepted
Can SHAP be used for linear mixed models?
In theory shapley values can be applied to any model, but I can't imagine a good reason to use shapley values when you are already using a linear (mixed) model.
Shapley values are associated with the ...
- 2,051
5
votes
What is Shapley value regression and how does one implement it?
There are two good papers to tell you a lot about the Shapley Value Regression:
Lipovetsky, S. (2006). Entropy criterion in logistic regression and Shapley value of predictors. Journal of Modern ...
- 51
5
votes
Accepted
characteristics of Shapley values
I assume you mean the SHAP paper and are referring to this chapter: Shapley Values.
You can basically find the answer in the Supplemental Material of the Lundberg paper: http://papers.nips.cc/paper/...
- 136
5
votes
SHAP algorithm for feature selecion
SHAP probably is not as useful as you would like.
In a keynote address to "Why R?", Frank Harrell discusses how feature selection is a mirage. While his simulations do not address SHAP in ...
- 67.2k
5
votes
How to determine relative contribution of explanatory variables in a linear regression
If $\mathcal{M}_1$ is your full model and you want to know the contribution of $X_1$ for example, then you fit another model $\mathcal{M}_2$ that includes everything but $X_1$. You can then compute a ...
- 14.3k
4
votes
Accepted
Reasons that LIME and SHAP might not agree with intuition
Just to state this up-front: most machine learning models just try to predict. They do not find/show causal effects, understand what is going on, model disease mechanisms or medical relationships. I.e....
- 35.2k
4
votes
Accepted
Why the contribution of a categorical value in SHAP trained on Catboost differs from observation to observation
There is a difference because the effect of the species ("target") is not independent of the other features: the GBM you have built has nontrivial (and perhaps even very complex) feature ...
- 5,051
4
votes
Accepted
Kernel SHAP - Estimation of contribution function
This sampling from marginals, which is done for many explanation methods, i.e. not only for SHAP, is heatedly debated (see Hooker et.al. arguing "against", see Janzing et.al. arguing "...
- 2,853
3
votes
Accepted
Why SHAP base/expected value is 0.5 for all my instances?
The baseline of Shapley values shown ($0.50$) is the average of all predictions. It is not a random base value. To quote from the original 2017 SHAP paper "A Unified Approach to Interpreting ...
- 46k
3
votes
SHAP values and feature-target correlations contradict each other - why?
What you described seems like a manifestation of Simpson's paradox. ML models usually have multiple interactions between their features so these "sign-reversal" phenomena might be associated ...
- 46k
3
votes
Accepted
SHAP values of Ensemble Model
To answer my own question: Yes, apparently linearity is one of the attributes of SHAP values. I found the answer in this github issue thread: enter link description here. The response I am referring ...
- 1,129
3
votes
Questions about the process of feature selection through feature importance
When you evaluate feature elimination strategies such as "remove features by importance until there are X features left" through cross-validation, X is a hyper-parameter of the training ...
- 35.2k
2
votes
Accepted
Use Shapley Values for explaining whole Data Frame instead of a Single prediction
The function itself isn't vectorized, but you can loop through your data explaining one instance at a time and then concatenate the instance-level lists into one data.frame.
Below is R code that ...
- 81
2
votes
Best way to assess SHAP values variability
Quite old issue without any replies, but I have a suggestion [here]: https://medium.com/@lucasramos_34338/visualizing-variable-importance-using-shap-and-cross-validation-bd5075e9063a:
Where I compute ...
- 121
2
votes
Shapley value vs ridge regression
There are several reasons why the two sets of values you computed are not the same:
$R_2$ contributions are not partial correlation or regression coefficients
The package you use to compute Shapley ...
- 3,162
2
votes
Accepted
Can adding more data, influence the SHAPley values?
One possible explanation: it depends what you mean by adding data because:
shapley value shows the individual contribution of a feature that is above or below the average contribution or in other ...
- 1,658
2
votes
Shap values on scaled dataset
As univariate transformation applied to a model’s inputs does not effect the Shapley values for the model, convert the explanation to the original feature space should just do the trick, see example ...
- 21
2
votes
Shapley values for groups of correlated features
See Relative variable importance/explained variation from a single model fit for a related discussion where a permutation-based importance measure breaks down with high collinearity but a new linear ...
- 98.5k
2
votes
Why does my neural network consider different features important compared to my decision tree?
I don't want to spread misinformation on this forum. So for future viewers who see this question I would like to apologize. There was an error in the code of the scaling function that I used for my ...
- 41
2
votes
Accepted
Why does the SHAP formula not state that |S| = |F|-1?
A key point to note about the Shapley Value is that it is also the average contribution feature $i$ makes to the fit metric/function $f()$ across all possible permutations of the $|F|$ features. Thus, ...
- 1,087
2
votes
Accepted
Can you sum the SHAP values of multiple lagged variables?
Basically this post from Stackoverflow answers my question:
"The short answer is yes, you can add up SHAP values across the columns to get the importance of a whole group of features (just make ...
Only top scored, non community-wiki answers of a minimum length are eligible