0
$\begingroup$

I am working on a Machine Learning model. One of the requests is to explain the models 'decisions' to the business. Therefore I am using Shapley Values (Game Theory).

I found an interesting example on the internet (https://cran.r-project.org/web/packages/iml/vignettes/intro.html), how we can Explain a Single prediction with Shapley (see code in R below). But this is only explaining one single prediction.

My questions:

  1. Is it possible to create one explanation for a whole data frame, instead of for just one single row? (for example: having the plot for the whole (test) data frame, instead of only one row).
  2. If yes, is it academically legit and how can I do that in R?

More information about Shapley values in R (https://www.rdocumentation.org/packages/iml/versions/0.9.0/topics/Shapley)

Thanks in advance!

Code

#Example Shapley
#https://cran.r-project.org/web/packages/iml/vignettes/intro.html

data("Boston", package  = "MASS")
head(Boston)

set.seed(42)
#install.packages("iml")
library("iml")
library("randomForest")
data("Boston", package  = "MASS")
rf = randomForest(medv ~ ., data = Boston, ntree = 50)

# We create a Predictor object, that holds the model and the data. 
# The iml package uses R6 classes: New objects can be created by calling Predictor$new()
X = Boston[which(names(Boston) != "medv")]
predictor = Predictor$new(rf, data = X, y = Boston$medv)

# Feature Importance
## Shifting each future, and measring how much the performance drops ## 
imp = FeatureImp$new(predictor, loss = "mae")
plot(imp)


# Shapley value. Assume that for 1 data point, the feature values play a game together, in which 
# they get the prediction as payout. Tells us how fairly distibute the payout among the feature values. 
View(X)
shapley = Shapley$new(predictor, x.interest = X[1,])
shapley$plot()

# Reuse the object to explain other data points 
shapley$explain(x.interest = X[2,])
shapley$plot()

# Results in data.frame form can be extracted like this: 
results = shapley$results
head(results)
$\endgroup$

closed as off-topic by Michael Chernick, Peter Flom May 1 at 10:30

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – Michael Chernick, Peter Flom
If this question can be reworded to fit the rules in the help center, please edit the question.

1
$\begingroup$

The function itself isn't vectorized, but you can loop through your data explaining one instance at a time and then concatenate the instance-level lists into one data.frame.

Below is R code that should work with your example (~3 min run time on a 16GB RAM laptop).

# Loop through the samples, explaining one instance at a time.
shap_values <- vector("list", nrow(X))  # initialize the results list.
system.time({
  for (i in seq_along(shap_values)) {
    set.seed(224)
    shap_values[[i]] <- iml::Shapley$new(predictor, x.interest = X[i, ],
                                         sample.size = 30)$results
    shap_values[[i]]$sample_num <- i  # identifier to track our instances.
  }
  data_shap_values <- dplyr::bind_rows(shap_values)  # collapse the list.
})

Is this approach legit? I can't think of a reason why it wouldn't be. I've seen Shapley value plots similar to partial dependence plots used in a variety of contexts. In addition to the awesome package iml, check out DALEX and--shameless plug--my package shapFlex (the overview vignette has plots) for this and other uses of Shapley values.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.