1
$\begingroup$

I found out that we have different solutions like below to explain the ML predictions

a) LIME

b) SHAP

Despite using all these approaches, I see that all of them work for certain data points and not work for certain data points. for example, let's assume we have a dataset with 6 re ords. LIME works/explanation makes sense for records 1,2,3 and 4 whereas LIME explanation doesn't make sense for records 5 an 6.

On the other hand, SHAP works well for 3,4,5 and 6 but doesn't explain well for records 1 and 2.

So, my question is below

machine learning model are not 100% explainable? How do you deal such scenarios when you wish to explain your predictions to the business users but you find out that local explanations make sense for few records and not for other records? Do you compromise on explanations/ go live even with some inconsistency in explanations?

Is it an expected behavior? Simply, what do you amswer when ypur business asks why certain explanations are inaccurate (and some are correct). How do we rely on such a solution?

Improve this question
$\endgroup$
13
  • 1
    $\begingroup$ You say "solutions like below" and mention specific records but there is no data attached to your question. Is that intentional? $\endgroup$
    – g g
    Feb 17, 2022 at 10:11
  • $\begingroup$ No, (nontrivial) machine learning models are not 100% explainable, not even closely. $\endgroup$
    – frank
    Feb 17, 2022 at 10:56
  • $\begingroup$ @gg - by solutions, I meant LIME amd SHAP. My question is on incosistency between different explainablw solutions amd within same solution. Meanimg, I would like to have one stable approach which can work well for all rows $\endgroup$
    – The Great
    Feb 17, 2022 at 11:59
  • $\begingroup$ @frank - when your business stakeholders, why certaim explanatioms are not accurate, what do we say? Wouldn't it reflect bad on the data scientist? $\endgroup$
    – The Great
    Feb 17, 2022 at 12:00
  • $\begingroup$ You usually cannot create a model that perfectly describes your data. Data scientists always try to strike the balance between the accuracy of the results and expenditure. Also, what do you think explainability of AI actually means? Do you explain noise? $\endgroup$
    – frank
    Feb 17, 2022 at 12:08

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.