I'm validating a model and as part of the process I've been calculating SHAP values for different validation datasets.
I've calculated SHAP values for every sample in each dataset taken their absolute values, summed, and normalised them. This allows me to see their aggregate contribution to the model predictions, as a percentage, for each dataset.
Although my top 20 features remain the same, I've noticed there are minor ranking shifts in their percentage contributions.
I believe this is caused by the over-emphasis or under-emphasis of patterns that the model has been trained to pick up, and which use specific sets of features. However, I'm not sure if this is a reasonable interpretation.
What causes shifts in the contribution of features, as measured by SHAP, on different datasets for the same model?