One way to explain individual predictions of an xgb classifier is to calculate contributions of each feature. To my knowledge there are two packages in R that can do this for you automatically. In the
xgboost package you can call the
predict.xgb.Booster function at set
TRUE. In the
xgboostExplainer package you can call the
Both methods result in a n-by-m dataframe where n = the number of observations and m = the number of features. However, in my hands the two methods give different values for the contributions of each prediction. These differences exceed floating point error, so it suggests different calculations are being performed. Interestingly, the difference in the final prediction values (sum of prediction contributions) is within floating point error.
Two specific questions:
Is there a preferred method within the stats community for calculating the contribution of each feature in an xgb classifier?
Why the discrepancy between the feature contributions between the two methods?
Reproducible example follows:
library(xgboost) library(xgboostExplainer) ## binary classification: data(agaricus.train, package='xgboost') data(agaricus.test, package='xgboost') train <- agaricus.train test <- agaricus.test bst <- xgboost(data = train$data, label = train$label, max_depth = 2, eta = 0.5, nthread = 2, nrounds = 5, objective = "binary:logistic") pred <- predict(bst, test$data) pred_contr <- predict(bst, test$data, predcontrib = TRUE) contr1 <- pred_contr[1,] xgb.train.data <- xgb.DMatrix(train$data, label = train$label) xgb.test.data <- xgb.DMatrix(test$data, label = test$label) explainer = buildExplainer(bst, xgb.train.data, type="binary", base_score = 0.5, trees = NULL) pred.breakdown = explainPredictions(bst, explainer, xgb.test.data) #discrepancy between prediction contributions breakdown1 <- pred.breakdown[1,] sum(contr1) - sum(breakdown1) summary(as.numeric(contr1) - as.numeric(breakdown1[1,])) ## Min. 1st Qu. Median Mean 3rd Qu. Max. #-0.2982 0.0000 0.0000 0.0000 0.0000 0.1712 #but not between final predictions summary(rowSums(pred_contr) - rowSums(pred.breakdown)) # Min. 1st Qu. Median Mean 3rd Qu. Max. ##-1.612e-07 -2.644e-08 -1.401e-08 2.783e-08 6.149e-08 3.226e-07