Gradient Boosting Machine

learn more… | top users | synonyms

0
votes
1answer
72 views

Reconciling boosted regression trees (BRT), generalized boosted models (GBM), and gradient boosting machine (GBM)

Questions: What is the difference(s) between boosted regression trees (BRT) and generalized boosted models (GBM)? Can they be used interchangeably? Is one a specific form of the other? Why did ...
1
vote
0answers
42 views

Plot the training and cross-validation deviance [closed]

I'm running the gbm model using R caret package. To tune the model I used 10-fold cross-validation. I tried to get the following plots to guide my model tuning but didn't succeed: Plot the training ...
0
votes
1answer
151 views

R gbm package variable influence

I'm using the excellent gbm package in R to do multinomial classification, and my question is about feature selection. After deciding the number of iterations ...
0
votes
1answer
64 views

Change settings in the prediction model (caret package)

I am using the package caret and GBM method for my predictions. ...
1
vote
1answer
110 views

GBM: Predict the response variable measured in {0,20}

I need to predict the response that has values in {0,20}. Should it be used as a factor or as a numeric value? How does it influence on the prediction error? I am using GBM with the Gaussian ...
0
votes
0answers
16 views

Which distribution should I better use to predict the response in {0,20} applying GBM? [duplicate]

I want to predict the response that is in {0,20}. I am using GBM to make the prediction. ...
1
vote
1answer
93 views

Boosted trees and Variable Interactions

How can one see in a Boosted trees classification model, which variables interact with each other and how much? I would like to make use o this in R gbm package if possible
0
votes
0answers
37 views

Using gbm in R with non-random missing data

The way gbm handles missing variables in R is by using surrogate splits. Is this appropriate to use when the data is not missing at random?
0
votes
0answers
26 views

GBM Prediction Interval Issue

I need to get prediction interval for GBM model (loss='ls'). I'm using this example as a basis http://scikit-learn.org/stable/auto_examples/ensemble/plot_gradient_boosting_quantile.html My model ...
0
votes
1answer
142 views

Training AUC and CV AUC in Boosted Regression Tree

My question is regarding the differences in the training data AUC score and the cross validation AUC score in boosted regression trees (BRT) built using the gbm.step function in the dismo package. I ...
0
votes
1answer
45 views

Different minimum observations per node in GBM model does not affect the AUC. How to explain?

My dataset is 6.3 million observations, with 150 features for each one. 25 000 of these observations are positive case and the rest is negative case, so about 1:250 class balance. I've been training ...
1
vote
0answers
142 views

Using an RMSE with derived confidence interval, to generate a prediction interval for an estimate

Previous questions have asked about creating prediction intervals for estimates derived from random forests or boosted regression trees, in a similar way to is easily achieved with linear regression ...
0
votes
0answers
107 views

Combining collinear continuous predictor variables in GBM

I'm dealing with a dataset (n=254) with one dependent variable (Y), and three independent variables (X1, X2, X3), all continuous. I would like to compare the contribution from each IV to Y. I've been ...
1
vote
0answers
83 views

How can I compare GBM feature importances to GBM partial dependence plots?

I am having trouble reconciling the difference between the indicated "importance" from a GBM that I am calculating with what is shown in the partial dependence plots. I would expect higher ...
0
votes
0answers
150 views

R: selecting appropriate fit statistics from GBM output

I'm using the gbm.step function in the dismo package in R to evaluate the contribution of three continuous variables on my ...
1
vote
3answers
401 views

R: partial dependency plots from GBM package. Values and y-axis

I'm using the gbm.step package in R to look at the influence of three continuous variables on my continuous response variable. I have 234 observations. The model: ...
0
votes
0answers
83 views

Parameter selection for GBM

I'm building a Gradient Boosting model. Given a dataset and event rate, is it possible to get a formula/ definitive strategy for the optimum number of trees, shrinkage parameter and depth of trees? I ...
0
votes
1answer
383 views

Generating PMML export of a gbm model in R?

Is it possible to generate PMML of a gbm model? When I try to use the pmml library, I get an error: Error in UseMethod("pmml") : no applicable method for 'pmml' applied to an object of class ...
1
vote
3answers
327 views

Performance drop between training and validation datasets

I have been using R's GBM (Gradient Boosting Machine) package for several months. I typically split my data into three partitions: Training, Validation, and Testing. I use the validation data set to ...
0
votes
2answers
196 views

How to get coefficients of gradient booting models?

I tried gradient boosting models using both gbm in R and sklearn in Python. However, neither of them can provide the ...
1
vote
2answers
102 views

Estimating expected lifetime from hazard ratio and estimated base hazard function

Apologies if this is a basic question, I am not very familiar with survival analysis ... I have trained a gradient boosted Cox proportional hazards model in R, and have been able to obtain reasonable ...
0
votes
1answer
165 views

GBM Bootstrap Prediction Interval Code Error

based on code presented in thread: How to find a GBM Prediction Interval I am trying to apply this to my dataset. Below is my full code, and I am having issues with the bootstrap function. ...
2
votes
1answer
393 views

How to find a GBM Prediction Interval

I am working with GBM models using the caret package and looking to find a method to solve the prediction intervals for my predicted data. I have searched extensively but only come up with a few ...
2
votes
0answers
141 views

Is multicollinearity a problem with gradient boosted trees (i.e. GBM)?

A question about multicollinearity for random forests has been asked and answered, but what about boosted trees?
0
votes
0answers
93 views

Pitfalls of using random forest/GBM on proportion data?

I have a set of data with a dependent variable which represents a proportion, and many of the samples contain a response of 1. I would like to build a random forest or GBM regression model on the ...
0
votes
1answer
43 views

Can adaboost choose the same variable for multiple splits for a given tree?

Can adaboost choose the same variable for multiple splits for a given tree? The model was given 100 + variable to choose from and it did choose them for the other trees in the ensemble. I am using ...
2
votes
0answers
129 views

How does GBM model handle categorical variables with many levels

I am using gbm model to fit a continuous dependent variable Y with several categorical variables, say, X, Z, V, and W. Suppose X has many levels (distinct values) and Y has moderate number of levels, ...
0
votes
0answers
40 views

GBM package: Why there is a missing node?

Why there is a missingNode as 3 as there are no missing values? I have the data in the following form: ...
0
votes
1answer
332 views

How to interpret the output of a multinomial classification model in R package gbm

After running a gradient boosted model with n data points using multinomial regression where the response variable (a factor, as required by the gbm function) has ...
0
votes
0answers
17 views

Finding the effects of certain levels of a factor predictor

I have fitted a binary classification gbm model, and one of the predictor variables, Affiliate has 50 different levels. Given ...
0
votes
0answers
110 views

Sample Weights for classification using Gradient-Boosted trees?

How can "weights" be given to different samples according to their relative importance while using Gradient boosted decision trees for classification? How does the ...
0
votes
0answers
340 views

gbm R multinomial vs bernoulli

I am using the gbm package to fit a binary variable using several attributes, some numeric and some categorical. Since the output varible was defined as factor I initially did ...
1
vote
2answers
145 views

gbm.perf with method = “test” returns n.trees from last run

As the title says, I'm getting some interesting results from gbm.perf. The first time I ran into trouble was after a run where n.trees was set to 7,000. When gbm.perf also returned 7,000 I got ...
0
votes
0answers
91 views

Estimating the running time of gbm grid tuning

I am trying to estimate the time it will take me to run a tuning grid on gbm (I am using the R Caret package but this is irrelevant as I am interested in the relative processing time). I can see many ...
0
votes
0answers
114 views

BRT analysis using count data

I have some problems with my BRT analysis. Introduction to the data: The dependent variable is count data of a specific palm species in SA, and the predictors consists of nine various kinds of ...
3
votes
2answers
527 views

How to find optimal values for the tuning parameters in boosting trees ?

I realise that there are 3 tuning parameters in the boosting trees model, i.e. the number of trees (number of iterations) shrinkage parameter number of splits (size of each constituent trees) My ...
1
vote
0answers
105 views

How do i estimate the Weights of the predictions assigned to each of the tree in GBM using R? How does GBM split nodes?

I ran a GBM model in R with loss function as bernoulli and n.trees=1000. I want to see the weights assigned to the predictions coming from 1000 trees. Is there any command in R that does that? How ...
1
vote
1answer
239 views

Negative predictions for binomial predictions from gbm in R

I've just fit a binomial model (training y = 0 or 1) using R's gbm package. When I calculated predicted values using my validation data, some of the predicted values were less than 0. Is this normal ...
1
vote
2answers
246 views

Combine decision trees from GBM to reduce output

I am curious if any research has been conducted to efficiently combine trees resulting from a gradient boosting process. I routinely run a process that generates 20 or 30 thousand trees in R. I then ...
1
vote
0answers
95 views

R: Which distribution to use with gbm for gamma distributed data?

When I use GLMs I can use the option family="Gamma" for analysing data consisting of positive real numbers. Also package gbm ...
9
votes
4answers
2k views

Why doesn't Random Forest handle missing values in predictors?

What are theoretical reasons to not handle missing values? Gradient boosting machines, regression trees handle missing values. Why doesn't Random Forest do that?
2
votes
1answer
306 views

How can I modify default parameters of a gbm.step plot?

I am using the function gbm.step() from the dismo package to assess the optimal number of boosting trees using k-fold cross ...
1
vote
0answers
53 views

Random Forest and Factor Predictors [duplicate]

How do decision tree based ensembles like random forest deal with categorical ("factor") predictor variables? My guess would be that indicator variables are created for each factor via a ...
2
votes
0answers
254 views

Binary Class Distribution Effects on Probability Scores - (gbm) Boosted Tree Regression Models

Any help would be greatly appreciated. Problem: I need help to better understand the probability scores that come from the result of a decision tree model. Specifically, I'm using the gbm package ...
3
votes
0answers
233 views

Could you explain how gradient boosting algorithm works?

I have read a lot about gbm in Greedy function Approximation: A Gradient boosting Machine (pdf), but I can't code the algorithm for example LS_Boost in a simple way. Can someone explain what $h(x;a)$ ...
2
votes
3answers
288 views

Heuristic Feature Selection for Gradient Boosting

I originally posted on Stack Overflow and was told to move it here. If I am trying to select from two different sets of features for a Gradient Boosting Machine, but I do not want to run through ...
0
votes
0answers
282 views

Measure the goodness-of-fit in boosted regression tree

What is the apropriate statistic to measure the goodness-of-fit in Boosted Regression Tree (or Gradient Boosting Regression) with continuous response? How can I calculate the coefficient of ...
1
vote
0answers
307 views

Significance of R Squared in Random Forest / GBM and GBM Tuning Parameters

I often get different level of responses when I discuss about R-Squared and its relevance to measuring the performance of a Random Forest or GBM model. In general, RMSE is a better and more ...
0
votes
1answer
234 views

Gradient boosting in R uses only a single variable

I am trying to build a boosting model using the package gbm in R. I have the following code: ...
1
vote
0answers
248 views

Mean Reciprocal Rank with GBM in R

Let's say I'm optimizing MRR with a GBM in R: ...