Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I'm using LIME to break down the observation for each row and am taking a look at the positive and negative factors that contribute to the probability outputted.
I filtered my dataset down to only records with a 95% or higher probability score, but when I look at the factors, all of them have a negative weight on the prediction. How can an observation have a 95% probability to be class 1 if all the factors are considered negative?
While somewhat unlikely this phenomenon can indeed happen. The results from local explainer by LIME can disagree (on occasion substantially) with the results of the global model. Probably it is worth considering different kernel widths as well as checking the goodness-of-fit of the LIME explainer too.
More details: LIME is training a model in the "neighbourhood" of the point we are trying to explain. There is no need for the global model (i.e. our overall ML model) and for our local model (i.e. the explainer trained by LIME) to be outputting the same results. They are two potentially very different models that are trained on different datasets. (The LIME explainer is trained on a perturbed version of the data close to our point of interest, the overall ML model uses all our training data.)
For this case in particular, as the focus are sample instances with very high probability, it is likely that all the neighbouring instances of them are of the same class too; the intercept (assuming we are training an LASSO regression model as an explainer) will be reasonably high and therefore that most of the features' factors are of negative weight. I would suggest trying different kernel widths so the neighbourhood size is varied. Finally, do check how good the LIME explainer is fitting the overall examples; while it unreasonable to expect great performance from it, it may happen that the explainer under-fits so substantially that any insights are misleading at best.
