13
$\begingroup$

I am reading up about two popular post hoc model interpretability techniques: LIME and SHAP

I am having trouble understanding the key difference in these two techniques.

To quote Scott Lundberg, the brains behind SHAP:

SHAP values come with the black box local estimation advantages of LIME, but also come with theoretical guarantees about consistency and local accuracy from game theory (attributes from other methods we unified)

I am having some trouble understanding what this 'theoretical guarantees about consistency and local accuracy from game theory' is. Since SHAP was developed after LIME, I assume it fills on some gaps which LIME fails to address. What are those?

Christoph Molnar's book in a chapter on Shapley Estimation states:

The difference between the prediction and the average prediction is fairly distributed among the features values of the instance - the shapley efficiency property. This property sets the Shapley value apart from other methods like LIME. LIME does not guarantee to perfectly distribute the effects. It might make the Shapley value the only method to deliver a full explanation

Reading this, I get a sense that SHAP is not a local but a glocal explanation of the data point. I could be wrong here and need some insight into what this above quote means. To summarize my ask: LIME produces Local explanations. How are SHAP's explanations different from LIME's?

$\endgroup$
  • 1
    $\begingroup$ Nice question (+1), I will try answering it when I get time but the obvious thing to notice is that LIME does not offer a globally consistent explanation while SHAP does. Also, SHAP definitely has been developed prior to LIME. SHAP builds heavily on Strumbelj & Kononenko's work from latE 00's/early 10's as well as works on economics on transferable utility cooperative games (e.g. Lipovetsky & Conklin (2001)). In addition, a lot of work on sensitivity analysis measurements (e.g. Sobol Indices) also goes that way. Core SHAP ideas were generally well-known prior to NIPS 2017. $\endgroup$ – usεr11852 Dec 1 '18 at 10:29
  • 1
    $\begingroup$ (Just to be clear regarding the second half of my comment above: I am not suggesting that the NIPS 2017 paper is the product of plagiarism or anything remotely like that. It is just that I often see people disregarding a lot of previous work on a field and treating a recent cross-section of the otherwise organic evolution of a field of knowledge as a major methodological breakthrough. In this particular case: algorithmic game theory has been around for decades as part of AI, just now it became semi-cool.) $\endgroup$ – usεr11852 Dec 1 '18 at 10:36
  • 1
    $\begingroup$ @usεr11852 Awaiting for your answer. Please do contribute your ideas on this. $\endgroup$ – user248884 Dec 5 '18 at 11:43
  • 2
    $\begingroup$ Haven't got all the necessary time yet; I have written about 400 words but it need at least another 6-7 hours of work as I have to reread some of papers and tighten my text - explaining SHAP without making over-simplifications is a bit challenging (for me at least). Probably I will make it before mid-December... :) $\endgroup$ – usεr11852 Dec 5 '18 at 22:29
  • 1
    $\begingroup$ @usεr11852 Sure. Will await :) $\endgroup$ – user248884 Dec 7 '18 at 13:50
4
$\begingroup$

LIME creates a surrogate model locally around the unit who's prediction you wish to understand. Thus it is inherently local. Shapely values 'decompose' the final prediction into the contribution of each attribute - this is what some mean by 'consistent' (the values add up to the actual prediction of the true model, this is not something you get with LIME). But to actually get the shapely values there is some decision that must be made about what to do/how to handle the values of the attributes 'left out', this is how the values are arrived at. In this decision there is some choice which could change the interpretation. If I 'leave out' an attribute do I average all the possibilities? Do choose some 'baseline'?

So Shapely actually tells you, in an additive way, how you got your score, but there is some choice about the 'starting point' (ie the decision about omitted attributes).

LIME simply tells you, in a local sense, what is the most important attribute around the data point of interest.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.