Can we use Shap to interpret output changes?

Question

Can we calculate the difference between Shapley values to interpret changes in the output? More precisely, if we get Shapley values for two different inputs, can we compare them to understand how much each feature led to the increase/decrease of the output?

Example: Let's say that I have a regression model with 10 features as input: $x=(x_{1}, x_{2}, x_{3}, ...,x_{10})$. I get two predictions: $y^{1}=70$ and $y^{2}=100$ and I want to understand how much each feature has led to such an increase (+30). Let's say that for the two inputs $x = (x_{1}, x_{2}, x_{3}, ..., x_{10})$ and $x' = ({x_{1}', x_{2}', x_{3}', ..., x_{10}'})$ I get the following Shapley values: $\phi = (7, -4, 6, 18, 7, 0, 14, -3, 17, 7)$ and $\phi' = (7, -6, 6, 20, 7, 0, 32, -3, 27, 9)$. Here I'm assuming that the expected value of all the prediction is 1 (this is why the Shapley values sum up to 69 and 99 respectively).

I calculate $\phi'-\phi = (\phi'_{1}-\phi_{1}, \phi'_{2}-\phi_{2}, ..., \phi'_{10}-\phi_{10})$ which in my example is going to be equal to: (0, -2, 0, 2, 0, 0, 18, 0, 10, 2).

• Is it correct to say that the 2nd feature led to a decrease of -2, the 4th feature led to an increase of 2, the 7th feature to an increase of 18, etc.?

I believe this can be inferred somehow from the property of the additive property of Shapley values but I'm not finding the connection yet.

Answer 1

Partly, this is correct.

• "led" implies causation, which is most probably not the case. Better say "is associated with".

• Thanks to SHAP's additivity properties, you can indeed explain the difference between two predictions by the sum of the differences between the SHAP values associated with the features.

It is fine to say: "The two predictions differ by 30. 2 of these 30 come from differences in feature $x_2$, ... and finally 2 from differences in feature $x_m$. However, such "effect" of $x_1$ would not generalize to other predictions. It only holds for comparing these specific predictions.