Gradient Boosting Machine

learn more… | top users | synonyms

0
votes
0answers
19 views

GBM Bootstrap Prediction Interval Code Error

based on code presented in thread: How to find a GBM Prediction Interval I am trying to apply this to my dataset. Below is my full code, and I am having issues with the bootstrap function. ...
2
votes
1answer
69 views

How to find a GBM Prediction Interval

I am working with GBM models using the caret package and looking to find a method to solve the prediction intervals for my predicted data. I have searched extensively but only come up with a few ...
0
votes
0answers
11 views

Is multicollinearity a problem with gradient boosted trees (i.e. GBM)?

A question about multicollinearity for random forests has been asked and answered, but what about boosted trees?
0
votes
0answers
24 views

Pitfalls of using random forest/GBM on proportion data?

I have a set of data with a dependent variable which represents a proportion, and many of the samples contain a response of 1. I would like to build a random forest or GBM regression model on the ...
0
votes
1answer
10 views

Can adaboost choose the same variable for multiple splits for a given tree?

Can adaboost choose the same variable for multiple splits for a given tree? The model was given 100 + variable to choose from and it did choose them for the other trees in the ensemble. I am using ...
1
vote
0answers
19 views

How does GBM model handle categorical variables with many levels

I am using gbm model to fit a continuous dependent variable Y with several categorical variables, say, X, Z, V, and W. Suppose X has many levels (distinct values) and Y has moderate number of levels, ...
0
votes
0answers
17 views

GBM package: Why there is a missing node?

Why there is a missingNode as 3 as there are no missing values? I have the data in the following form: ...
0
votes
0answers
31 views

How to interpret the output of a multinomial classification model in R package gbm

After running a gradient boosted model with n data points using multinomial regression where the response variable (a factor, as required by the gbm function) has ...
0
votes
0answers
11 views

Finding the effects of certain levels of a factor predictor

I have fitted a binary classification gbm model, and one of the predictor variables, Affiliate has 50 different levels. Given ...
0
votes
0answers
32 views

Sample Weights for classification using Gradient-Boosted trees?

How can "weights" be given to different samples according to their relative importance while using Gradient boosted decision trees for classification? How does the ...
0
votes
0answers
88 views

gbm R multinomial vs bernoulli

I am using the gbm package to fit a binary variable using several attributes, some numeric and some categorical. Since the output varible was defined as factor I initially did ...
1
vote
1answer
35 views

gbm.perf with method = “test” returns n.trees from last run

As the title says, I'm getting some interesting results from gbm.perf. The first time I ran into trouble was after a run where n.trees was set to 7,000. When gbm.perf also returned 7,000 I got ...
0
votes
0answers
22 views

Estimating the running time of gbm grid tuning

I am trying to estimate the time it will take me to run a tuning grid on gbm (I am using the R Caret package but this is irrelevant as I am interested in the relative processing time). I can see many ...
0
votes
0answers
55 views

BRT analysis using count data

I have some problems with my BRT analysis. Introduction to the data: The dependent variable is count data of a specific palm species in SA, and the predictors consists of nine various kinds of ...
3
votes
2answers
155 views

How to find optimal values for the tuning parameters in boosting trees ?

I realise that there are 3 tuning parameters in the boosting trees model, i.e. the number of trees (number of iterations) shrinkage parameter number of splits (size of each constituent trees) My ...
1
vote
0answers
54 views

How do i estimate the Weights of the predictions assigned to each of the tree in GBM using R? How does GBM split nodes?

I ran a GBM model in R with loss function as bernoulli and n.trees=1000. I want to see the weights assigned to the predictions coming from 1000 trees. Is there any command in R that does that? How ...
1
vote
1answer
90 views

Negative predictions for binomial predictions from gbm in R

I've just fit a binomial model (training y = 0 or 1) using R's gbm package. When I calculated predicted values using my validation data, some of the predicted values were less than 0. Is this normal ...
1
vote
2answers
90 views

Combine decision trees from GBM to reduce output

I am curious if any research has been conducted to efficiently combine trees resulting from a gradient boosting process. I routinely run a process that generates 20 or 30 thousand trees in R. I then ...
1
vote
0answers
39 views

R: Which distribution to use with gbm for gamma distributed data?

When I use GLMs I can use the option family="Gamma" for analysing data consisting of positive real numbers. Also package gbm ...
4
votes
3answers
326 views

Why doesn't Random Forest handle missing values in predictors?

What are theoretical reasons to not handle missing values? Gradient boosting machines, regression trees handle missing values. Why doesn't Random Forest do that?
1
vote
1answer
83 views

How can I modify default parameters of a gbm.step plot?

I am using the function gbm.step() from the dismo package to assess the optimal number of boosting trees using k-fold cross ...
1
vote
0answers
47 views

Random Forest and Factor Predictors [duplicate]

How do decision tree based ensembles like random forest deal with categorical ("factor") predictor variables? My guess would be that indicator variables are created for each factor via a ...
0
votes
0answers
73 views

modeling rates with machine learning tools (svm, gbm, nnet)

I have a numeric integer variable that is knowly proportional to an exposure measure plus other continuous / categorical covariates. If I were to use classical log-linear glms i would model ...
2
votes
0answers
140 views

Binary Class Distribution Effects on Probability Scores - (gbm) Boosted Tree Regression Models

Any help would be greatly appreciated. Problem: I need help to better understand the probability scores that come from the result of a decision tree model. Specifically, I'm using the gbm package ...
2
votes
0answers
141 views

Could you explain how gradient boosting algorithm works?

I have read a lot about gbm in Greedy function Approximation: A Gradient boosting Machine (pdf), but I can't code the algorithm for example LS_Boost in a simple way. Can someone explain what $h(x;a)$ ...
0
votes
0answers
29 views

Discretization of the split in random forest estimation

If the dataset I'm working with has too many independent variable dimensions, I may need to choose a proper subset of all possible locations in my feature vector to determine whether I am going to use ...
2
votes
2answers
145 views

Heuristic Feature Selection for Gradient Boosting

I originally posted on Stack Overflow and was told to move it here. If I am trying to select from two different sets of features for a Gradient Boosting Machine, but I do not want to run through ...
0
votes
0answers
39 views

Representing a class by negative number for training in GBM in R

I have already generated a huge train file of ~1GB in which the ground truth is one of 4 classes {-1,0,1,2}. Yes, I agree that I could have done it using 3 instead of -1, but for some conventional ...
0
votes
0answers
166 views

Measure the goodness-of-fit in boosted regression tree

What is the apropriate statistic to measure the goodness-of-fit in Boosted Regression Tree (or Gradient Boosting Regression) with continuous response? How can I calculate the coefficient of ...
1
vote
0answers
203 views

Significance of R Squared in Random Forest / GBM and GBM Tuning Parameters

I often get different level of responses when I discuss about R-Squared and its relevance to measuring the performance of a Random Forest or GBM model. In general, RMSE is a better and more ...
0
votes
1answer
175 views

Gradient boosting in R uses only a single variable

I am trying to build a boosting model using the package gbm in R. I have the following code: ...
1
vote
0answers
156 views

Mean Reciprocal Rank with GBM in R

Let's say I'm optimizing MRR with a GBM in R: ...
2
votes
1answer
584 views

Partial Dependency plots and Gradient boosting (GBM package)

Is it possible to plot a partial dependency plot to display the class probability and estimate the effects of a predictor for a GBM model? Something similar to ...
1
vote
0answers
113 views

How does gbm determine the missing node if the training set is complete?

I have a training set which is complete - no missing values, but yet if I pretty print one of the trees in gbm I see that the algorithm has determined a strategy for missing data. This is smarter ...
1
vote
0answers
68 views

Accounting for seasonality when using gradient boosting

I'm a novice attempting to predict automobile sales using a combination of previous sales (seasonal AR model), macroeconomic indicators such as CPI, consumer sentiment index etc. and more ...
2
votes
2answers
403 views

How to choose the number of trees in a generalized boosted regression model?

Is there a strategy for choosing the number of trees in a GBM? Specifically, the ntrees argument in R's ...
1
vote
0answers
45 views

How to approach prediction of new observations with incomplete data from model built from complete data

I currently have a gradient boosting model that uses the gbm package in R that classifies observations at the end of a year. Daily behaviors are logged for each ...
3
votes
1answer
83 views

How to communicate regression model performance to non-stats people?

In R I created a gradient boosted random forest from 100,000 records with 10 cross validation folds using the gbm library. I want to communicate the strength and ...
0
votes
1answer
239 views

Effect of features that are highly correlated with each other on a decision tree

I have a dataset of roughly 500 features and am training a binary classifier using GBM - gradient boosted machines, an ensemble of decision trees. Of these 500 variables, I am sure some are highly ...
0
votes
1answer
102 views

Negative value prediction using gradient bosting with Gaussian distribution in gbm

Why does gbm specified with Gaussian distribution throw negative values? Can it be controlled or modified if one does not want negative predictions (in ...
2
votes
1answer
97 views

What is the difference between the values in the 'fit' attribute of a gbm object and values computed by gbm.predict?

My intuition is that the fitted values and predicted values of a gbm object should be identical. But in this example with just one tree, the values are different: ...
4
votes
2answers
834 views

How does cross validation work in R's gbm package?

Can someone provide a work flow about this? For instance, suppose I am doing binary classification, For each iteration of the algorithm: Randomly sample k*N rows, where k is the bag.fraction, and N ...
1
vote
0answers
250 views

Machine Learning Algorithms vs. Linear Regression

Do machine learning algorithms like Boosted Regression Trees (in the R package (gbm)) follow the same statistical assumptions of not including correlated predictor variables in GLM? i.e. If I have ...
2
votes
1answer
511 views

has anyone implemented an autoencoder with random forests

I'm interested in exploring autoencoders which can be used to develop a compressed representation of data useful for machine learning. In my experience random forests are easier to work with and more ...
0
votes
0answers
87 views

Model Matrix and GBM

I am using the GBM package to create a prediction model. Can GBM handle categorical predictors, or do I need to create a model matrix before I input ?
4
votes
3answers
2k views

How to use R gbm with distribution = “adaboost”?

Documentation states that R gbm with distribution = "adaboost" can be used for 0-1 classification problem. Consider the following code fragment: ...
2
votes
1answer
695 views

GBM relative influence

For a regression problem, I used gradient boosting machines and assessed RMSE. My dataset is comprised of 34 features and 10,000 records. Only 2 predictors were considered 'important' (importance for ...
4
votes
1answer
840 views

Strategy to set the GBM parameters

I would like to know how you would set the parameters for the GBM model? 1) I could optimize the parameters sequentially. At first, using a large value for shrinkage and a small number of iterations ...
7
votes
2answers
902 views

Classification with GBM in R and imbalanced class sizes

I'm dealing with a supervised binary classification issue. I'd like to use the GBM package to classify individuals as uninfected/infected. I have 15 times more uninfected than infected individuals. I ...
1
vote
1answer
292 views

Does the 'fit' attribute of a gbm object contain the OOB estimates?

I have looked at the help (?gbm) and documentation for gbm.object, it only says that fit: a vector containing the fitted values on the scale of regression function (e.g. log-odds ...