9
$\begingroup$

I tried gradient boosting models using both gbm in R and sklearn in Python. However, neither of them can provide the coefficients of the model. For gbm in R, it seems one can get the tree structure, but I can't find a way to get the coefficients. For sklearn in Python, I can't even see the tree structure, not to mention the coefficients. Can anyone give me some help?

After searching online for couple of hours, I still can't find the answer. I can find similar questions since 2009, but no answers. Like the followings:

This make me wonder if R and Python are mainly used by academic people, and thus majority of the users don't care about how to use them in industry. For example, if you want to implement the results in some real-time platform which doesn't run Python, what would you do?

Improve this question
$\endgroup$
2
  • $\begingroup$ {pmml} or {h20} packages may be useful. As pretty.gbm.tree from {gbm}. But none sounds quite what you're looking for $\endgroup$
    – charles
    Commented Oct 4, 2014 at 3:03
  • $\begingroup$ If you fit a gradient boosting model using gbm() and put the result in gbm1, you should be able to see the structure by typing str(gbm1). You can access the elements as needed. In this case, gbm1 is a glm.object --- the documentation describes its structure. $\endgroup$
    – jvbraun
    Commented Dec 6, 2014 at 4:20

4 Answers 4

Reset to default
8
$\begingroup$

I use R "in industry". GBM's and other tree-based methods don't have "coefficients" so it's pointless to try to extract them.

What you CAN do is encode each tree as a SQL query. It take a little effort, but once you can do it for a single tree, you can loop over all the trees in a model, generate ~500 SQL queries, and use them to score your model on a database of your choosing.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ +1 for making the SQl analogy. BTW, do you know any tools can translate the tree into SQL? $\endgroup$
    – Haitao Du
    Commented Feb 12, 2018 at 16:54
  • 1
    $\begingroup$ @hxd1011 I've never seen a package to do this, but after some google, it looks like there's a gist on github you could try: gist.github.com/shanebutler/5456942 $\endgroup$
    – Zach
    Commented Feb 16, 2018 at 14:14
1
$\begingroup$

Like Zach mentioned earlier, "coefficients" don't really apply for a GBM. I'm not sure how you're implementing it, but in a package like CARET (for R) you can look at variable importance during model building. You can also see something similar in the vignette for the GBM package in R. In the GBM package, I think it is called relative influence; the maths behind it is in the 2001 paper by Friedman.

Both of those approaches would go some way to giving you a ranking of how "useful"/"important" your variables were in classifying the target using a GBM.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ This is a useful point, but I interpret from the OP's final sentence that they want to use the model in something else, & so they want the coefficients for that, rather than to determine variable importance. $\endgroup$
    – gung - Reinstate Monica
    Commented Mar 4, 2015 at 4:47
  • 2
    $\begingroup$ Variable importance is not the analogous concept, partial dependency plots are. $\endgroup$
    – Matthew Drury
    Commented Feb 12, 2018 at 17:26
0
$\begingroup$

Use PMML to transfer the model to other platforms, assuming your other platform is PMML-compatible.

Improve this answer
$\endgroup$
0
$\begingroup$

For python, you could use import _pickle as cPickle to save the model to a pickle file and restore the model from the pickle file.

The codes for store the model:

with open("gbmFit.pkl", "wb") as pickle_file:
    cPickle.dump(model, pickle_file)

For restore the model, one could use following code:

with open('gbmFit.pkl', 'rb') as pickle_file:
    gbmfit = cPickle.load(pickle_file)
    y_pred_restore = gbmfit.predict(np.array(x1 + 1).reshape((-1, 1)))

Now you could save the model and save it for later real time use.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.