1
$\begingroup$

This post is kind of related to these two posts here and here

I learnt that model expected value (average prediction) is nothing but performance of the model when all input features are absent.

But I see that this expected value is calculated from a model with input features (whose values are not zero). But it is interpreted as model's simplest performance when X is not present.

I find this difficult to understand. Because, we compute mean(y_pred) from a model that we build using X input features (and their values are not zero).You can refer the code sample from this answer where the average is computed using mean(y_pred). And this is how SHAP package does it as well.

we are taking average from a model which has non-zero values for all predictors? Can I know why is it done this way?

I see that in SHAP average/expected values are calculated this way only,

Can you help me understand why is it done this way?

$\endgroup$
10
  • 2
    $\begingroup$ It is hard to understand what you mean, especially since the two linked questions seem to be completely unrelated. Could you try to clarify? Maybe you could use an example? What do you mean by "performance" of the model? The unclear part is also that you are talking about "expected value at X=0 with X != 0" what doesn't make any sense at all. The linked threads didn't make any of the claims that you mention. $\endgroup$
    – Tim
    Apr 1, 2022 at 12:06
  • $\begingroup$ Those threads were created by me as well. Those threads talk about what is expected value and how it is calculated. In this thread, based on what I learnt from other thread (especially SHAP post), you can see that the average prediction is computed based on average of y_pred. Do I make sense till here? before I go on to explain further? $\endgroup$
    – The Great
    Apr 1, 2022 at 12:14
  • $\begingroup$ @Tim - I removed model performance part from the post. Hope it provides improved clarity now $\endgroup$
    – The Great
    Apr 1, 2022 at 12:16
  • $\begingroup$ By X = 0- i mean all input features' value is 0. For the model that we build X not equal 0 (because they have non-zero values for all features) $\endgroup$
    – The Great
    Apr 1, 2022 at 12:23
  • 2
    $\begingroup$ Features cannot be at the same time 0 and not 0... Neither of the answers you refer to makes such claims as mentioned by your question... $\endgroup$
    – Tim
    Apr 1, 2022 at 12:25

2 Answers 2

3
$\begingroup$

First of all, the SHAP paper does not mention features being all zeros, but "would be predicted if we did not know any features" and that is not the same. It would be the same for linear regression where if in

$$ y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \dots + \beta_k X_k + \varepsilon $$

you would set $X_1 = X_2 = \dots = X_k = 0$, the only thing that would remain is $\beta_0$, it would just predict the "average" $y$ value. It wouldn't necessarily be true for other models, but the SHAP paper mentions this by noticing that "most models cannot handle arbitrary patterns of missing input values".

Why does averaging the predictions tell you about the performance of the model as if the features were unknown? The details are given in the paper, but for intuition, you can think of it as integrating out the features. If your model $f$ approximates the conditional expectation

$$ E[y|X] = f(X) $$

then by the law of total expectation, we know that

$$ E[y] = E[E[y|X]] = \int_X E[y|X] \,dP $$

So if you average over the features, you are left with the marginal prediction. If you average over the features, it is like you didn't use the features. Of course, this is just an approximation, because the $X's$ you observed in the data are not necessarily representative of the whole distribution of $X$'s, but we make the assumption that our data is representative every time we use machine learning.

$\endgroup$
11
  • $\begingroup$ Thanks for your help. upvoted. I am no expert in stats. But in your response, when you mention you would set X1=X2=⋯=Xk=0 , Is it incorrect to call this as X=0? $\endgroup$
    – The Great
    Apr 1, 2022 at 13:42
  • $\begingroup$ I am asking only to correct my mistake and improve $\endgroup$
    – The Great
    Apr 1, 2022 at 13:43
  • $\begingroup$ @TheGreat it's just a matter of notation, but you could write it like this. But as said, using zeroes there would work for linear models and not necessarily for other models. $\endgroup$
    – Tim
    Apr 1, 2022 at 13:44
  • 1
    $\begingroup$ @TheGreat if you have a binary feature X where males are coded as 1, then X=0 means "female" not "absence of sex". So it is not the same. $\endgroup$
    – Tim
    Apr 1, 2022 at 13:52
  • 1
    $\begingroup$ @TheGreat no and 0.5 value has nothing to do with the feature being unknown. It is not about averaging the features, but the predictions. If averaging the features would be enough, we wouldn't need things like SHAP. $\endgroup$
    – Tim
    Apr 1, 2022 at 14:16
2
$\begingroup$

I was of the understanding that absence of features, mean X=0.

The SHapley Additive Explanations (SHAP) expresses the difference in the prediction of some value, when the $X_i$ is known in comparison to when the $X_i$ is not known.

Not knowing the feature is different from the absence of the feature.

The 'absence of features' or better 'not knowing the feature' needs to be defined/considered carefully. In the context of SHAP it doesn't meant that $X_i = 0$ but it means that we do not know the value of $X_i$. (we still may know the distribution of potential values of $X_i$ or we could estimate this distribution based on the data)

In order to make a prediction, without knowing the value of $X_i$, one would use the expectation of the outcome when $X_i$ is unknown (we compute the expectation of the potential outcome over the known or estimated distribution of $X_i$). Using $X_i = 0$ is not a good way to make a prediction when $X_i$ is unknown.

$\endgroup$
1
  • $\begingroup$ Thanks for your help. Upvoted. $\endgroup$
    – The Great
    Apr 1, 2022 at 13:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.