0
$\begingroup$

This is the formula for fitting a boosted regression trees from gbm package:

gbm(formula = formula(data),
    distribution = "gaussian",
    data = list(),
    weights,
    var.monotone = NULL,
    n.trees = 100,
    interaction.depth = 1,
    n.minobsinnode = 10,
    shrinkage = 0.001,
    bag.fraction = 0.5,
    train.fraction = 1.0,
    cv.folds = 0,
    keep.data = TRUE,
   verbose = "CV",
  class.stratify.cv=NULL,
  n.cores = NULL)

After fitting the model, I can use it to predict values in a new dataset using:

predict(object,
        newdata, 
        n.trees,
        type="link",
        single.tree=FALSE,
       ...)

Is there some way to write it down in some form of ifelse statement? For e.g. if I fit a linear regression in R:

  mdl <- lm(y ~ x1 + x2)
  summary(mdl)

With the printed summary, I can write the original equation like so:

   y = intercept + beta1.x1 + beta2.x2 

using the information from the summary above and store this equation in an excel sheet or some other software which will take the vector of x1 and x2 and the predict y.

How can I do the same for an object fitted in gbm? The reason I want to do this is because I want to write the fitted equation in a software.

$\endgroup$

closed as off-topic by Michael Chernick, mdewey, Alexis, Robert Long, DeltaIV Jun 18 '18 at 10:42

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – Michael Chernick, mdewey, Alexis, Robert Long, DeltaIV
If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ Well, it's a weighted average of 100 trees, so I'm not sure you'd really want to do that. And what will you do if you rerun it and get slightly different results, as happens with GBMs and Random Forests? Or construct GBMs with thousands of trees? Why not leave it in R in the form of the model object? $\endgroup$ – jbowman Jun 13 '18 at 14:53
  • $\begingroup$ Okay. This is what I am doing at the moment. However, the issue is that I need to put the calibrated gbm implementation in another software and that's I wanted to code it down. $\endgroup$ – 89_Simple Jun 13 '18 at 15:55
  • $\begingroup$ In addition, I understand the stochastic part of is driven by the bag.fraction argument. I am fitting the model with a set seed or even bag.fraction = 1 such that my results are not same everytime it is run $\endgroup$ – 89_Simple Jun 13 '18 at 15:57
  • $\begingroup$ Can you list your source and destination software? $\endgroup$ – EngrStudent Jun 14 '18 at 16:40
  • $\begingroup$ I am using R to fit the model but destination software is still something I do not know. Most likely I will use R but instead of the model saved as object, I want to write this as an equation. However, the thing is I am fitting a regression type of model and gbm. With regression, I can simply write y = intercept + slopex1 + slopex2 but for gbm, I do not know yet $\endgroup$ – 89_Simple Jun 14 '18 at 20:01
1
$\begingroup$

It can be done, but doesn't come out in a nice "if-then-else" format. The function pretty.gbm.tree takes a gbm model object and an integer referring to which tree you want to see and creates a table with all the relevant information:

data(iris)
iris.mod <- gbm(Species ~ ., distribution="multinomial", data=iris,
                 n.trees=2000, shrinkage=0.01, cv.folds=5,
                 verbose=FALSE, n.cores=1)
pretty.gbm.tree(iris.mod, i.tree=1)

gives the following nice table:

  SplitVar SplitCodePred LeftNode RightNode MissingNode ErrorReduction Weight Prediction
0        2        2.6000        1         2           3          16.32     75    -0.0006
1       -1        0.0300       -1        -1          -1           0.00     24     0.0300
2       -1       -0.0150       -1        -1          -1           0.00     51    -0.0150
3       -1       -0.0006       -1        -1          -1           0.00     75    -0.0006

The documentation explains how it works, but basically, LeftNode tells you which row to go to for a left branch, and similarly for RightNode and MissingNode. SplitCodePred tells you the value which you split on if the node splits (with SplitVar telling you which variably you split on) or the prediction if the node is a leaf node (SplitVar == -1).

You could loop over all the indices of the trees and construct a data frame containing the results, then work with that. It seems to me that a heap is the natural way to go. (The output from pretty.gbm.tree is just a standard data frame, so you can rbind them all together, adjusting the values of the LeftNode etc. appropriately.) You would then have to combine that with your shrinkage / learning rate to get something that, hopefully, would reproduce the tree predictions.

$\endgroup$
  • 1
    $\begingroup$ Make sure you write unit tests!!! $\endgroup$ – Matthew Drury Jun 13 '18 at 17:19
  • 2
    $\begingroup$ @MatthewDrury what do you mean by unit tests? $\endgroup$ – 89_Simple Jun 13 '18 at 17:31
  • 2
    $\begingroup$ It's a technique for ensuring correctness of code which would be very, very useful in an exercise like this: en.wikipedia.org/wiki/Unit_testing $\endgroup$ – Matthew Drury Jun 13 '18 at 18:19

Not the answer you're looking for? Browse other questions tagged or ask your own question.