10
$\begingroup$

Let's say I have prediction for an observation with 3 shap values: -2, 3 and 5 for feature A, B and C respectively. Then I scale the absolute value of the shap values so they sum to 1 (i.e A=0.2, B=0.3 and C=0.5). Is it appropriate to interpret these scaled shap values as percent contribution to the prediction? For example, view feature A as having a 20% contribution to the prediction.

$\endgroup$
3
  • $\begingroup$ Welcome to CV.SE. Is this a regression or classification task? $\endgroup$
    – usεr11852
    Dec 16, 2020 at 2:38
  • $\begingroup$ it's a classification task. $\endgroup$
    – tom1919
    Dec 16, 2020 at 17:21
  • $\begingroup$ Thank you for the clarification. Yes, you are mostly fine. Good job (+1). Please see my answer below I expand on this a bit more. $\endgroup$
    – usεr11852
    Dec 17, 2020 at 1:00

1 Answer 1

8
$\begingroup$

Almost yes. There are a few caveats regarding directly interpreting the scaled SHAP values, as the percentage contributions of our final classification prediction for a single observation.

  1. Raw SHAP values for classification tasks are often shown as additive contribution in the log-odds domain. That means that it is not a linear scale we are dealing with and scaling them will not massively simplify that. (Some implementation transform them to probabilities but that's not always the case.)

  2. SHAP values' baseline is always relative based on the average of all predictions; i.e. the contribution (SHAP value) of feature X regards the difference between the actual prediction and the mean prediction. Therefore they do not fully explain how a singe prediction came to be. (This may vary a bit based on the implementation but worth checking.)

  3. Depending on the learner used if the features A, B and C are on different scales, their coefficient values in the log-odds domain might not be directly comparable. (Usually not a big problem because often the features are binned when it comes to feature importance and/or we pre-process the data but it can happen.)

  4. SHAP (and Shapley) values are approximations of the model's behaviour. They are not guarantee to account perfectly on how a model works. (Obvious point but sometimes forgotten.)

The above being noted, what you describe (summing the absolute SHAP values for an individual prediction and normalising them to get percentage of contribution) is reasonable. It is actually how we calculate overall feature importances with SHAP values. In this more general case, we sum the absolute values per feature across all our observations (and potentially normalise them afterwards). These overall feature contributions will be against our mean prediction of course; as we are examining relative importance for a classifier's overall output this is perfectly fine. For a single point it can be argued that any of the caveats mentioned above partially invalidates our proposed interpretation but realistically the contribution of each feature would "thereabouts" in terms of percentages for the prediction of a single observation.

$\endgroup$
4
  • $\begingroup$ Thanks for the thorough response! The only concern I had was with caveat #2 and the other ones don’t impact my specific case (but good points!). My rationale for not including the mean prediction was because its contribution across predictions is the same and not interesting to report. If I include a footnote stating that the estimated percent contributions are calculated after removing the common denominator of the mean prediction, do you think there’s any other concern with respect to this caveat? $\endgroup$
    – tom1919
    Dec 17, 2020 at 19:06
  • 1
    $\begingroup$ I am glad I could help. No, you will be fine. As mentioned your proposed approach does hold water; clarifying little caveats can stop any strong criticism so otherwise you are good. If this answer is helpful please consider upvoting it and if it resolves your question marking it as the accepted answer. $\endgroup$
    – usεr11852
    Dec 17, 2020 at 19:43
  • $\begingroup$ The answer was most definitely helpful and I completely agree but I unfortunately don't have enough points to vote. Once i do, I'll be sure to come back here and upvote this. Thanks again! $\endgroup$
    – tom1919
    Dec 17, 2020 at 19:48
  • 1
    $\begingroup$ I was referring to main post, I think you refer to the clarification comment. :) $\endgroup$
    – usεr11852
    Dec 17, 2020 at 21:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.