2
$\begingroup$

I'm trying to use LIME to locally explain black-box models predictions. It works by generating 5000 neighbors for an instance by perturbing the features. Then it does forward-selection by iteratively fitting a simple Ridge model over that local neighborhood, where the label is the black-box model prediction. After n iterations, the selected features are considered as the most important for the black-box decision, and the weights are indicative of how important.

I'm measuring how good is that explanation by measuring the R^2 of the local model.

When the black-box model is LogisticRegression, the average R^2 is ~0.65 which I consider pretty good. However, once I switched to a more sophisticated RandomForest black-box model, I'm getting a very low (~0.06) mean R^2 score.

Why is that? does that mean that the LIME method isn't good for explaining the surface of my model predictions?

Technicalities: I have 6 features -age, gender, hemoglobin_last, hemoglobin_slope, creatinine_last and creatinine_slope and the outcome I'm trying to predict is chronic_renal_failure. The data is pretty imbalanced: 1.5M controls and 5K cases. For measuring quality I randomly selected 100 cases and 100 controls and averaged the R^2 score of their explanation.

(related to Interpret predictions of black box models)

Improve this question
$\endgroup$
1
  • $\begingroup$ I am getting similarly low results trying to explain my SVM classifier, haven't compared it to to another model though. Here it is suggested to adjust the kernel width of the local model (decrease to make the fit more local). This seems to definitely change the $R^2$ values for me. Don't really know why the values differ so much for different models... Will post more if I get some more insights into it. $\endgroup$
    – František Kaláb
    Commented Feb 6, 2018 at 16:41

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.