1
$\begingroup$

I am using SHAP values in a model explainability analysis and I am seeing a pattern of results where the SHAP values are clearly pointing in a positive or negative direction for most features, i.e. feature values are highly correlated with SHAP values. I interpret this as evidence of directional relationships between features and predictions.

Now the interesting part: I also calculated correlations between feature variables and the target variable, and in some cases the sign of the correlation flips - indicating the opposite relationship.

In short: the correlation between feature values and SHAP values and the correlation between feature values and target values are in disagreement.

E.g.: r(feature, SHAP) = 0.71 r(feature, target) = -0.22

I have trouble interpreting this pattern. Any ideas?

$\endgroup$

1 Answer 1

3
$\begingroup$

What you described seems like a manifestation of Simpson's paradox. ML models usually have multiple interactions between their features so these "sign-reversal" phenomena might be associated with confounding variables biasing the effect measurement. If you have a reproducible example it would be easier to comment further. I would also suggest looking at PDPs (partial dependency plots) both the "standard" version we get through marginalisation of feature x as well as the ones we get via SHAP (in later case go forward and colour-code individual point by the outcome so any underlying pattern is more obvious). They might include relevant information. Finally, maybe it is worth trying a "glass-box" model (e.g. GAMs, EBMs, etc.) such that the influence of feature x to the outcome is immediately quantifiable.

$\endgroup$
3
  • $\begingroup$ What is the difference between the partial R^2 and the SHAP Value for a linear regression model? (interpretation) $\endgroup$
    – skan
    Commented Nov 13, 2022 at 20:21
  • 1
    $\begingroup$ That is a great question as they have some obvious qualitative similarities. It is a rather involved one for the comment section though, so feel free to create a stand-alone question! :) $\endgroup$
    – usεr11852
    Commented Nov 13, 2022 at 22:43
  • $\begingroup$ I have just created the question in the datascience section. I don't know if it's better to use the crossvalidation or stats section. datascience.stackexchange.com/questions/116160/… Feel free to reply or to improve the question itself (English is not my language). $\endgroup$
    – skan
    Commented Nov 14, 2022 at 13:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.