36
$\begingroup$

I am reading up about two popular post hoc model interpretability techniques: LIME and SHAP

I am having trouble understanding the key difference in these two techniques.

To quote Scott Lundberg, the brains behind SHAP:

SHAP values come with the black box local estimation advantages of LIME, but also come with theoretical guarantees about consistency and local accuracy from game theory (attributes from other methods we unified)

I am having some trouble understanding what this 'theoretical guarantees about consistency and local accuracy from game theory' is. Since SHAP was developed after LIME, I assume it fills on some gaps which LIME fails to address. What are those?

Christoph Molnar's book in a chapter on Shapley Estimation states:

The difference between the prediction and the average prediction is fairly distributed among the features values of the instance - the shapley efficiency property. This property sets the Shapley value apart from other methods like LIME. LIME does not guarantee to perfectly distribute the effects. It might make the Shapley value the only method to deliver a full explanation

Reading this, I get a sense that SHAP is not a local but a glocal explanation of the data point. I could be wrong here and need some insight into what this above quote means. To summarize my ask: LIME produces Local explanations. How are SHAP's explanations different from LIME's?

Improve this question
$\endgroup$
7
  • 5
    $\begingroup$ Nice question (+1), I will try answering it when I get time but the obvious thing to notice is that LIME does not offer a globally consistent explanation while SHAP does. Also, SHAP definitely has been developed prior to LIME. SHAP builds heavily on Strumbelj & Kononenko's work from latE 00's/early 10's as well as works on economics on transferable utility cooperative games (e.g. Lipovetsky & Conklin (2001)). In addition, a lot of work on sensitivity analysis measurements (e.g. Sobol Indices) also goes that way. Core SHAP ideas were generally well-known prior to NIPS 2017. $\endgroup$
    – usεr11852
    Dec 1, 2018 at 10:29
  • 3
    $\begingroup$ (Just to be clear regarding the second half of my comment above: I am not suggesting that the NIPS 2017 paper is the product of plagiarism or anything remotely like that. It is just that I often see people disregarding a lot of previous work on a field and treating a recent cross-section of the otherwise organic evolution of a field of knowledge as a major methodological breakthrough. In this particular case: algorithmic game theory has been around for decades as part of AI, just now it became semi-cool.) $\endgroup$
    – usεr11852
    Dec 1, 2018 at 10:36
  • 1
    $\begingroup$ @usεr11852 Awaiting for your answer. Please do contribute your ideas on this. $\endgroup$
    – user248884
    Dec 5, 2018 at 11:43
  • 2
    $\begingroup$ Haven't got all the necessary time yet; I have written about 400 words but it need at least another 6-7 hours of work as I have to reread some of papers and tighten my text - explaining SHAP without making over-simplifications is a bit challenging (for me at least). Probably I will make it before mid-December... :) $\endgroup$
    – usεr11852
    Dec 5, 2018 at 22:29
  • 2
    $\begingroup$ @usεr11852 Please do put up your answer when possible. I understand it must be intense to distill concepts about two pretty complicated topics. Still, to get the discussion started, please do post whatever you have composed. Appreciate your help :) $\endgroup$
    – user248884
    Dec 24, 2018 at 7:03

1 Answer 1

Reset to default
21
$\begingroup$

  • LIME creates a surrogate model locally around the unit whose prediction you wish to understand. Thus it is inherently local.

  • Shapley values 'decompose' the final prediction into the contribution of each attribute - this is what some mean by 'consistent' (the values add up to the actual prediction of the true model, and this is not something you get with LIME).

But to actually get the shapley values, some decision must be made about what to do/how to handle the values of the attributes 'left out', which is how the values are arrived at. In this decision, there is some choice which could change the interpretation. If I 'leave out' an attribute, do I average all the possibilities? Do choose some 'baseline'?

in a nutshell:

Shapley values actually tell you, in an additive way, how you got your score, but there is some choice about the 'starting point' (i.e. the decision about omitted attributes).

LIME simply tells you, in a local sense, what is the most important attribute around the data point of interest.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Can you also add how each model make their score (e.g. shap score) - I found these scores quite annoying as they are not normalize and I don't understand what they mean ! $\endgroup$
    – Areza
    Nov 28, 2019 at 8:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.