4
$\begingroup$

I tried gradient boosting models using both gbm in R and sklearn in Python. However, neither of them can provide the coefficients of the model. For gbm in R, it seems one can get the tree structure, but I can't find a way to get the coefficients. For sklearn in Python, I can't even see the tree structure, not to mention the coefficients. Can anyone give me some help?

After searching online for couple of hours, I still can't find the answer. I can find similar questions since 2009, but no answers. Like the followings:

This make me wonder if R and Python are mainly used by academic people, and thus majority of the users don't care about how to use them in industry. For example, if you want to implement the results in some real-time platform which doesn't run Python, what would you do?

$\endgroup$
  • $\begingroup$ {pmml} or {h20} packages may be useful. As pretty.gbm.tree from {gbm}. But none sounds quite what you're looking for $\endgroup$ – charles Oct 4 '14 at 3:03
  • $\begingroup$ If you fit a gradient boosting model using gbm() and put the result in gbm1, you should be able to see the structure by typing str(gbm1). You can access the elements as needed. In this case, gbm1 is a glm.object --- the documentation describes its structure. $\endgroup$ – jvbraun Dec 6 '14 at 4:20
5
$\begingroup$

I use R "in industry". GBM's and other tree-based methods don't have "coefficients" so it's pointless to try to extract them.

What you CAN do is encode each tree as a SQL query. It take a little effort, but once you can do it for a single tree, you can loop over all the trees in a model, generate ~500 SQL queries, and use them to score your model on a database of your choosing.

$\endgroup$
  • $\begingroup$ +1 for making the SQl analogy. BTW, do you know any tools can translate the tree into SQL? $\endgroup$ – Haitao Du Feb 12 '18 at 16:54
  • $\begingroup$ @hxd1011 I've never seen a package to do this, but after some google, it looks like there's a gist on github you could try: gist.github.com/shanebutler/5456942 $\endgroup$ – Zach Feb 16 '18 at 14:14
1
$\begingroup$

Like Zach mentioned earlier, "coefficients" don't really apply for a GBM. I'm not sure how you're implementing it, but in a package like CARET (for R) you can look at variable importance during model building. You can also see something similar in the vignette for the GBM package in R. In the GBM package, I think it is called relative influence; the maths behind it is in the 2001 paper by Friedman.

Both of those approaches would go some way to giving you a ranking of how "useful"/"important" your variables were in classifying the target using a GBM.

$\endgroup$
  • 1
    $\begingroup$ This is a useful point, but I interpret from the OP's final sentence that they want to use the model in something else, & so they want the coefficients for that, rather than to determine variable importance. $\endgroup$ – gung Mar 4 '15 at 4:47
  • 1
    $\begingroup$ Variable importance is not the analogous concept, partial dependency plots are. $\endgroup$ – Matthew Drury Feb 12 '18 at 17:26
0
$\begingroup$

Use PMML to transfer the model to other platforms, assuming your other platform is PMML-compatible.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.