I'm playing with LIME to explain the prediction of a machine learning model.
LIME trains a (locally weighted) linear surrogate model around a point of interest. The weights of that surrogate model are the feature importances of your model's prediction at that point.
However, it's not clear to me how to interpret the intercept term. It's the "base" prediction if all features are zero, which seems meaningless.
(Compare this to SHAP, where the "base" value is the average prediction across the training set.)
So, does it make sense to run LIME without using an intercept term? This way, the feature importances will directly sum up to the prediction.
If not, which I suspect, how do I interpret the intercept term?