I am quite new to the space, but I was wondering if it was possible to do causal analysis with a model trained with Gradient Boosting of decision trees?

I know there are packages like causalnex, https://github.com/quantumblacklabs/causalnex, but it does not seem to support gradient boosting, as it is a completely different framework.



For gradient boosting I am assuming you mean gradient boosting a linear model. The issue with this is that you are 'regularizing' your coefficients. So essentially you shrink them. Causal interpretation of regularized coefficients from a traditional POV is pretty controversial for well defined regularization techniques like LASSO and Ridge and I have not heard of any efforts to do it with boosting. Most practitioners would just do Ridge from a bayesian perspective and I would assume the model would be fairly similar or better performing than the boosted one.

With that said, it is possible to carry around the coefficients and standard errors and adjust them after each boosting round by simply adding the coefficients and updating your standard error. This will get you a p-value in the end but I doubt it is technically correct.

Some old code I have actually does this in a time-series setting where I combine decomposition (splitting a time series into trend + seasonality + extra factors) with boosting and I provide a normal ols summary for the extra factors if given. See the 'Dealing with Exogenous Variables' section to see this: enter image description here

But like I said, if you want to use regularization I would just stick with Ridge and go from there.

  • $\begingroup$ Thanks @Tylerr for the answer. What about for a causal model of gradient boosting of decision trees? $\endgroup$ – rockie May 26 at 14:29
  • $\begingroup$ You could look at BART: cran.r-project.org/web/packages/BART/BART.pdf . $\endgroup$ – Tylerr May 26 at 14:47
  • $\begingroup$ thanks a lot !! $\endgroup$ – rockie May 26 at 15:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.