1
$\begingroup$

If I build a random forest model (classifier) and display the results in a partial dependence plot (PDP) and observe a clear abrupt change. Is it then reasonable to extract the split-points of every tree from a particular predictor to say something about the highest density (mean/median), with the range (i.e. 2.5% and 97.5%) accompanied by a histogram, where these split-points are placed. I want to quantify something about the observed changes in the PDP and not only describe them. Although, I understand that if I would have a non-linear response, i.e wave function, this does not really help. Yet, if I simply observe a sharp increase or decrease can this be reasonable? Furthermore, with large amount of predictors the range can be extremely large, but this is also useful information, which can indicate that the changes are less abrupt. For the data I use this seems to quite well reflect the observations in the PDPs. What are your thoughts on this?

Thank you in advance.

#Something like this

set.seed(123)
data          <- data.frame(y=as.factor(c(rbinom(500, 1, .95), rbinom(500, 1, .5))), x=c(1:1000))

weight        <- min(as.data.frame(table(data$y))[2])

mod           <- randomForest::randomForest(y~x, data=data, weight=weight, strat=data$y, ntree=100, nodesize=weight*.2)

par(mfrow = c(1,2))
partialPlot(mod, data, x.var = x)

splits        <- list()
for(n in 1:100){
  extsplit    <- getTree(mod, k=n, labelVar = T)
  ext         <- extsplit$`split point`[extsplit$`split var` == "x"]
  splits[[n]] <- ext[!duplicated(ext)]}

splitloc1   <- do.call(rbind.data.frame, splits)
splitloc2   <- as.numeric((tidyr::gather(splitloc1)[,2]))

mean(splitloc2 ,na.rm = T)
quantile(splitloc2, c(.025, .975), na.rm = T)

hist(splitloc2)
$\endgroup$
1
$\begingroup$

I like where this is going.

The day with the split is it is a function of the subsample of observations seen for that predictor in each tree. Getting the range of the values of those splits would give you an idea of whether there is true signal or just noise in where the split occurs.

In order to not confound the results, perhaps you can limit your search to only root-splits since that is the only way to guarantee that no other confounder (or feature interaction) may modify the location of the split.

You might want to look into https://github.com/slundberg/shap and treeExplainer in how they then seek to interpret the differences in groups across the split.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.