So I know there is a feature_importances_ variable under the XGBoost classifier. I was wondering if there is a way to see the deciding features for each observation? This will allow me to understand why the machine learning algorithm predicted its class for each observation.
I looked into LIME: https://github.com/marcotcr/lime
And it looks to have what I need. I'm just curious if there are any other libraries that do something similar.