Assuming I have a classification problem where I have binary dependent variable Y and independent variables X1-X10. The X variables are categorical.

Say we are interested in the marginal effect of X1. I understand that marginal effect is the derivative of fitted probability over variable of interest X1.

Assume that X1 is categorical with 3 levels

1) in the case of logistic regression model, the exponential of coefficients of the 2 levels (1 level as reference) of X1 can be interpreted as marginal effect of X1 at the 2 levels comparing to the reference level, holding all other X variables (X2-X10) constant.

2) If we were to use a tree-based model instead, say boosted tree, according to documentation of R plot.gbm:

Plots the marginal effect of the selected variables by "integrating" out the other variables.

The integration method is based on: https://statweb.stanford.edu/~jhf/ftp/trebst.pdf

My question is, when interpreting the marginal effect of X1 using the boosted tree method, can we still say the effect (based on the partial dependence plot) is the marginal effect of X1, holding all the other variables constant (same as in logistic regression)? I'm still trying to wrap my head around the Friedman paper.

Second question is, if we simply plot fitted value Y-hat against X1, maybe averaging Y-hat within each category of X1, how do we interpret the resulting graph? can we say it's the marginal effect of X1, holding all other variables constant? My guess is no, since it's just the average fitted probability plotted against X1.

Also, do we have a similar package for calculating marginal effect when using the xgboost model? so far, I've been simply plotting the fitted/predicted value against X1, this probably shows how influential X1 is, but I don't think this is really marginal effect controlling all other X variables.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.