2
$\begingroup$

I'm using LIME to break down the observation for each row and am taking a look at the positive and negative factors that contribute to the probability outputted.

I filtered my dataset down to only records with a 95% or higher probability score, but when I look at the factors, all of them have a negative weight on the prediction. How can an observation have a 95% probability to be class 1 if all the factors are considered negative?

$\endgroup$
1
  • $\begingroup$ Fun little question (+1). I think what you report, a manifestation of some of the short-coming around the overall "local surrogate models" explainer methodology; please see my answer below for more details. :) $\endgroup$
    – usεr11852
    May 25, 2020 at 22:36

1 Answer 1

2
$\begingroup$

While somewhat unlikely this phenomenon can indeed happen. The results from local explainer by LIME can disagree (on occasion substantially) with the results of the global model. Probably it is worth considering different kernel widths as well as checking the goodness-of-fit of the LIME explainer too.

More details: LIME is training a model in the "neighbourhood" of the point we are trying to explain. There is no need for the global model (i.e. our overall ML model) and for our local model (i.e. the explainer trained by LIME) to be outputting the same results. They are two potentially very different models that are trained on different datasets. (The LIME explainer is trained on a perturbed version of the data close to our point of interest, the overall ML model uses all our training data.)

For this case in particular, as the focus are sample instances with very high probability, it is likely that all the neighbouring instances of them are of the same class too; the intercept (assuming we are training an LASSO regression model as an explainer) will be reasonably high and therefore that most of the features' factors are of negative weight. I would suggest trying different kernel widths so the neighbourhood size is varied. Finally, do check how good the LIME explainer is fitting the overall examples; while it unreasonable to expect great performance from it, it may happen that the explainer under-fits so substantially that any insights are misleading at best.

$\endgroup$
4
  • $\begingroup$ Thanks for the in-depth explanation. What is a good range for the kernel_width to experiment with? $\endgroup$
    – Jon
    May 25, 2020 at 23:04
  • $\begingroup$ Cool, I am glad I could help. By default, the kernel width is 0.75 times the square root of the number of features. I would use increment of that (say [1.0, 1.5, 2.5, ...], etc.), unfortunately there is no hard advice here. The wider our kernel, the more "global" our LIME explainer will be. $\endgroup$
    – usεr11852
    May 26, 2020 at 8:12
  • $\begingroup$ @usεr11852 - I have a Lime related problem. Would you be interested to help me with it? stats.stackexchange.com/questions/569621/… $\endgroup$
    – The Great
    Mar 29, 2022 at 15:54
  • $\begingroup$ @TheGreat Sure, I will check it in the following day or two. $\endgroup$
    – usεr11852
    Mar 29, 2022 at 16:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.