Gradient Boosting Machine

learn more… | top users | synonyms

1
vote
1answer
20 views

Intuitive explanations of differences between Gradient Boosting Trees (GBM) & Adaboost

I'm trying to understand the differences between GBM & Adaboost. These are what I've understood so far: There are both boosting algorithms, which learns from previous model's errors and ...
1
vote
0answers
33 views

Regression Tree Predictions

For various regression tree algorithms (e.g. GBM, Random Forest, Extra Trees), is there any sensible way to get predictions for new data when the independent variables for the new cases are much ...
0
votes
1answer
30 views

Repeating the same Random Forests through gbm is inconsistent?

I've noticed that when running a piece multiple times, the gbm produced (see below) produces slightly different results when viewing the summary. Should that be expected? I.e. running gbm regression ...
1
vote
0answers
33 views

Using gbm to eliminate variables before glm

I have a classification problem I am attempting to model using logistic regression (via the glm package in R): ...
2
votes
1answer
44 views

Choosing a sample rate for GBM models

I've created several GBM models to tune the parameters (trees, shrinkage and depth) to my data and the model performs well on the out-of-time sample. The data is credit card transactions (running into ...
1
vote
0answers
33 views

Formula to derive probabilities when using 'gbm' method through 'caret' package in R

I am training a classification problem by 'gbm' algorithm through 'caret' package in r.The response variable is a yes/nope type. Here 'objmodel' is the model I trained through method='gbm' and package ...
0
votes
0answers
77 views

Caret - Changing tuneGrid gives different results for same parameters

I'm having an issue training a GBM model in caret. I'm quite new to all of this, but I'll try to explain things as best as possible, but please let me know if you need any further info. My code looks ...
0
votes
1answer
252 views

Reconciling boosted regression trees (BRT), generalized boosted models (GBM), and gradient boosting machine (GBM)

Questions: What is the difference(s) between boosted regression trees (BRT) and generalized boosted models (GBM)? Can they be used interchangeably? Is one a specific form of the other? Why did ...
1
vote
0answers
65 views

Plot the training and cross-validation deviance [closed]

I'm running the gbm model using R caret package. To tune the model I used 10-fold cross-validation. I tried to get the following plots to guide my model tuning but didn't succeed: Plot the training ...
0
votes
1answer
296 views

R gbm package variable influence

I'm using the excellent gbm package in R to do multinomial classification, and my question is about feature selection. After deciding the number of iterations ...
0
votes
1answer
157 views

Change settings in the prediction model (caret package)

I am using the package caret and GBM method for my predictions. ...
1
vote
1answer
156 views

GBM: Predict the response variable measured in {0,20}

I need to predict the response that has values in {0,20}. Should it be used as a factor or as a numeric value? How does it influence on the prediction error? I am using GBM with the Gaussian ...
0
votes
0answers
16 views

Which distribution should I better use to predict the response in {0,20} applying GBM? [duplicate]

I want to predict the response that is in {0,20}. I am using GBM to make the prediction. ...
1
vote
2answers
139 views

Boosted trees and Variable Interactions

How can one see in a Boosted trees classification model, which variables interact with each other and how much? I would like to make use o this in R gbm package if possible
0
votes
0answers
60 views

Using gbm in R with non-random missing data

The way gbm handles missing variables in R is by using surrogate splits. Is this appropriate to use when the data is not missing at random?
0
votes
0answers
46 views

GBM Prediction Interval Issue

I need to get prediction interval for GBM model (loss='ls'). I'm using this example as a basis http://scikit-learn.org/stable/auto_examples/ensemble/plot_gradient_boosting_quantile.html My model ...
0
votes
1answer
210 views

Training AUC and CV AUC in Boosted Regression Tree

My question is regarding the differences in the training data AUC score and the cross validation AUC score in boosted regression trees (BRT) built using the gbm.step function in the dismo package. I ...
0
votes
1answer
68 views

Different minimum observations per node in GBM model does not affect the AUC. How to explain?

My dataset is 6.3 million observations, with 150 features for each one. 25 000 of these observations are positive case and the rest is negative case, so about 1:250 class balance. I've been training ...
1
vote
0answers
187 views

Using an RMSE with derived confidence interval, to generate a prediction interval for an estimate

Previous questions have asked about creating prediction intervals for estimates derived from random forests or boosted regression trees, in a similar way to is easily achieved with linear regression ...
0
votes
0answers
130 views

Combining collinear continuous predictor variables in GBM

I'm dealing with a dataset (n=254) with one dependent variable (Y), and three independent variables (X1, X2, X3), all continuous. I would like to compare the contribution from each IV to Y. I've been ...
1
vote
0answers
96 views

How can I compare GBM feature importances to GBM partial dependence plots?

I am having trouble reconciling the difference between the indicated "importance" from a GBM that I am calculating with what is shown in the partial dependence plots. I would expect higher ...
0
votes
0answers
189 views

R: selecting appropriate fit statistics from GBM output

I'm using the gbm.step function in the dismo package in R to evaluate the contribution of three continuous variables on my ...
1
vote
3answers
584 views

R: partial dependency plots from GBM package. Values and y-axis

I'm using the gbm.step package in R to look at the influence of three continuous variables on my continuous response variable. I have 234 observations. The model: ...
0
votes
0answers
92 views

Parameter selection for GBM

I'm building a Gradient Boosting model. Given a dataset and event rate, is it possible to get a formula/ definitive strategy for the optimum number of trees, shrinkage parameter and depth of trees? I ...
0
votes
1answer
472 views

Generating PMML export of a gbm model in R?

Is it possible to generate PMML of a gbm model? When I try to use the pmml library, I get an error: Error in UseMethod("pmml") : no applicable method for 'pmml' applied to an object of class ...
1
vote
3answers
400 views

Performance drop between training and validation datasets

I have been using R's GBM (Gradient Boosting Machine) package for several months. I typically split my data into three partitions: Training, Validation, and Testing. I use the validation data set to ...
1
vote
2answers
261 views

How to get coefficients of gradient booting models?

I tried gradient boosting models using both gbm in R and sklearn in Python. However, neither of them can provide the ...
1
vote
2answers
120 views

Estimating expected lifetime from hazard ratio and estimated base hazard function

Apologies if this is a basic question, I am not very familiar with survival analysis ... I have trained a gradient boosted Cox proportional hazards model in R, and have been able to obtain reasonable ...
0
votes
1answer
216 views

GBM Bootstrap Prediction Interval Code Error

based on code presented in thread: How to find a GBM Prediction Interval I am trying to apply this to my dataset. Below is my full code, and I am having issues with the bootstrap function. ...
4
votes
1answer
482 views

How to find a GBM Prediction Interval

I am working with GBM models using the caret package and looking to find a method to solve the prediction intervals for my predicted data. I have searched extensively but only come up with a few ...
2
votes
0answers
171 views

Is multicollinearity a problem with gradient boosted trees (i.e. GBM)?

A question about multicollinearity for random forests has been asked and answered, but what about boosted trees?
0
votes
0answers
110 views

Pitfalls of using random forest/GBM on proportion data?

I have a set of data with a dependent variable which represents a proportion, and many of the samples contain a response of 1. I would like to build a random forest or GBM regression model on the ...
0
votes
1answer
54 views

Can adaboost choose the same variable for multiple splits for a given tree?

Can adaboost choose the same variable for multiple splits for a given tree? The model was given 100 + variable to choose from and it did choose them for the other trees in the ensemble. I am using ...
2
votes
0answers
169 views

How does GBM model handle categorical variables with many levels

I am using gbm model to fit a continuous dependent variable Y with several categorical variables, say, X, Z, V, and W. Suppose X has many levels (distinct values) and Y has moderate number of levels, ...
1
vote
0answers
46 views

GBM package: Why there is a missing node?

Why there is a missingNode as 3 as there are no missing values? I have the data in the following form: ...
0
votes
1answer
431 views

How to interpret the output of a multinomial classification model in R package gbm

After running a gradient boosted model with n data points using multinomial regression where the response variable (a factor, as required by the gbm function) has ...
0
votes
0answers
17 views

Finding the effects of certain levels of a factor predictor

I have fitted a binary classification gbm model, and one of the predictor variables, Affiliate has 50 different levels. Given ...
0
votes
0answers
131 views

Sample Weights for classification using Gradient-Boosted trees?

How can "weights" be given to different samples according to their relative importance while using Gradient boosted decision trees for classification? How does the ...
1
vote
2answers
191 views

gbm.perf with method = “test” returns n.trees from last run

As the title says, I'm getting some interesting results from gbm.perf. The first time I ran into trouble was after a run where n.trees was set to 7,000. When gbm.perf also returned 7,000 I got ...
3
votes
2answers
738 views

How to find optimal values for the tuning parameters in boosting trees ?

I realise that there are 3 tuning parameters in the boosting trees model, i.e. the number of trees (number of iterations) shrinkage parameter number of splits (size of each constituent trees) My ...
1
vote
0answers
117 views

How do i estimate the Weights of the predictions assigned to each of the tree in GBM using R? How does GBM split nodes?

I ran a GBM model in R with loss function as bernoulli and n.trees=1000. I want to see the weights assigned to the predictions coming from 1000 trees. Is there any command in R that does that? How ...
2
votes
1answer
305 views

Negative predictions for binomial predictions from gbm in R

I've just fit a binomial model (training y = 0 or 1) using R's gbm package. When I calculated predicted values using my validation data, some of the predicted values were less than 0. Is this normal ...
2
votes
2answers
294 views

Combine decision trees from GBM to reduce output

I am curious if any research has been conducted to efficiently combine trees resulting from a gradient boosting process. I routinely run a process that generates 20 or 30 thousand trees in R. I then ...
1
vote
0answers
121 views

R: Which distribution to use with gbm for gamma distributed data?

When I use GLMs I can use the option family="Gamma" for analysing data consisting of positive real numbers. Also package gbm ...
9
votes
4answers
3k views

Why doesn't Random Forest handle missing values in predictors?

What are theoretical reasons to not handle missing values? Gradient boosting machines, regression trees handle missing values. Why doesn't Random Forest do that?
2
votes
1answer
351 views

How can I modify default parameters of a gbm.step plot?

I am using the function gbm.step() from the dismo package to assess the optimal number of boosting trees using k-fold cross ...
1
vote
0answers
53 views

Random Forest and Factor Predictors [duplicate]

How do decision tree based ensembles like random forest deal with categorical ("factor") predictor variables? My guess would be that indicator variables are created for each factor via a ...
2
votes
0answers
296 views

Binary Class Distribution Effects on Probability Scores - (gbm) Boosted Tree Regression Models

Any help would be greatly appreciated. Problem: I need help to better understand the probability scores that come from the result of a decision tree model. Specifically, I'm using the gbm package ...
3
votes
0answers
259 views

Could you explain how gradient boosting algorithm works?

I have read a lot about gbm in Greedy function Approximation: A Gradient boosting Machine (pdf), but I can't code the algorithm for example LS_Boost in a simple way. Can someone explain what $h(x;a)$ ...
2
votes
3answers
324 views

Heuristic Feature Selection for Gradient Boosting

I originally posted on Stack Overflow and was told to move it here. If I am trying to select from two different sets of features for a Gradient Boosting Machine, but I do not want to run through ...