Gradient Boosting Machine

learn more… | top users | synonyms

0
votes
1answer
8 views

Can adaboost choose the same variable for multiple splits for a given tree?

Can adaboost choose the same variable for multiple splits for a given tree? The model was given 100 + variable to choose from and it did choose them for the other trees in the ensemble. I am using ...
1
vote
0answers
13 views

How does GBM model handle categorical variables with many levels

I am using gbm model to fit a continuous dependent variable Y with several categorical variables, say, X, Z, V, and W. Suppose X has many levels (distinct values) and Y has moderate number of levels, ...
0
votes
0answers
17 views

GBM package: Why there is a missing node?

Why there is a missingNode as 3 as there are no missing values? I have the data in the following form: ...
0
votes
0answers
27 views

How to interpret the output of a multinomial classification model in R package gbm

After running a gradient boosted model with n data points using multinomial regression where the response variable (a factor, as required by the gbm function) has ...
0
votes
0answers
11 views

Finding the effects of certain levels of a factor predictor

I have fitted a binary classification gbm model, and one of the predictor variables, Affiliate has 50 different levels. Given ...
0
votes
0answers
25 views

Sample Weights for classification using Gradient-Boosted trees?

How can "weights" be given to different samples according to their relative importance while using Gradient boosted decision trees for classification? How does the ...
0
votes
0answers
84 views

gbm R multinomial vs bernoulli

I am using the gbm package to fit a binary variable using several attributes, some numeric and some categorical. Since the output varible was defined as factor I initially did ...
1
vote
1answer
32 views

gbm.perf with method = “test” returns n.trees from last run

As the title says, I'm getting some interesting results from gbm.perf. The first time I ran into trouble was after a run where n.trees was set to 7,000. When gbm.perf also returned 7,000 I got ...
0
votes
0answers
22 views

Estimating the running time of gbm grid tuning

I am trying to estimate the time it will take me to run a tuning grid on gbm (I am using the R Caret package but this is irrelevant as I am interested in the relative processing time). I can see many ...
0
votes
0answers
53 views

BRT analysis using count data

I have some problems with my BRT analysis. Introduction to the data: The dependent variable is count data of a specific palm species in SA, and the predictors consists of nine various kinds of ...
3
votes
2answers
149 views

How to find optimal values for the tuning parameters in boosting trees ?

I realise that there are 3 tuning parameters in the boosting trees model, i.e. the number of trees (number of iterations) shrinkage parameter number of splits (size of each constituent trees) My ...
1
vote
0answers
53 views

How do i estimate the Weights of the predictions assigned to each of the tree in GBM using R? How does GBM split nodes?

I ran a GBM model in R with loss function as bernoulli and n.trees=1000. I want to see the weights assigned to the predictions coming from 1000 trees. Is there any command in R that does that? How ...
1
vote
1answer
85 views

Negative predictions for binomial predictions from gbm in R

I've just fit a binomial model (training y = 0 or 1) using R's gbm package. When I calculated predicted values using my validation data, some of the predicted values were less than 0. Is this normal ...
1
vote
2answers
86 views

Combine decision trees from GBM to reduce output

I am curious if any research has been conducted to efficiently combine trees resulting from a gradient boosting process. I routinely run a process that generates 20 or 30 thousand trees in R. I then ...
1
vote
0answers
38 views

R: Which distribution to use with gbm for gamma distributed data?

When I use GLMs I can use the option family="Gamma" for analysing data consisting of positive real numbers. Also package gbm ...
4
votes
3answers
317 views

Why doesn't Random Forest handle missing values in predictors?

What are theoretical reasons to not handle missing values? Gradient boosting machines, regression trees handle missing values. Why doesn't Random Forest do that?
1
vote
1answer
76 views

How can I modify default parameters of a gbm.step plot?

I am using the function gbm.step() from the dismo package to assess the optimal number of boosting trees using k-fold cross ...
1
vote
0answers
47 views

Random Forest and Factor Predictors [duplicate]

How do decision tree based ensembles like random forest deal with categorical ("factor") predictor variables? My guess would be that indicator variables are created for each factor via a ...
0
votes
0answers
72 views

modeling rates with machine learning tools (svm, gbm, nnet)

I have a numeric integer variable that is knowly proportional to an exposure measure plus other continuous / categorical covariates. If I were to use classical log-linear glms i would model ...
2
votes
0answers
134 views

Binary Class Distribution Effects on Probability Scores - (gbm) Boosted Tree Regression Models

Any help would be greatly appreciated. Problem: I need help to better understand the probability scores that come from the result of a decision tree model. Specifically, I'm using the gbm package ...
2
votes
0answers
136 views

Could you explain how gradient boosting algorithm works?

I have read a lot about gbm in Greedy function Approximation: A Gradient boosting Machine (pdf), but I can't code the algorithm for example LS_Boost in a simple way. Can someone explain what $h(x;a)$ ...
0
votes
0answers
29 views

Discretization of the split in random forest estimation

If the dataset I'm working with has too many independent variable dimensions, I may need to choose a proper subset of all possible locations in my feature vector to determine whether I am going to use ...
2
votes
2answers
140 views

Heuristic Feature Selection for Gradient Boosting

I originally posted on Stack Overflow and was told to move it here. If I am trying to select from two different sets of features for a Gradient Boosting Machine, but I do not want to run through ...
0
votes
0answers
39 views

Representing a class by negative number for training in GBM in R

I have already generated a huge train file of ~1GB in which the ground truth is one of 4 classes {-1,0,1,2}. Yes, I agree that I could have done it using 3 instead of -1, but for some conventional ...
0
votes
0answers
160 views

Measure the goodness-of-fit in boosted regression tree

What is the apropriate statistic to measure the goodness-of-fit in Boosted Regression Tree (or Gradient Boosting Regression) with continuous response? How can I calculate the coefficient of ...
1
vote
0answers
201 views

Significance of R Squared in Random Forest / GBM and GBM Tuning Parameters

I often get different level of responses when I discuss about R-Squared and its relevance to measuring the performance of a Random Forest or GBM model. In general, RMSE is a better and more ...
0
votes
1answer
174 views

Gradient boosting in R uses only a single variable

I am trying to build a boosting model using the package gbm in R. I have the following code: ...
1
vote
0answers
151 views

Mean Reciprocal Rank with GBM in R

Let's say I'm optimizing MRR with a GBM in R: ...
2
votes
1answer
568 views

Partial Dependency plots and Gradient boosting (GBM package)

Is it possible to plot a partial dependency plot to display the class probability and estimate the effects of a predictor for a GBM model? Something similar to ...
1
vote
0answers
111 views

How does gbm determine the missing node if the training set is complete?

I have a training set which is complete - no missing values, but yet if I pretty print one of the trees in gbm I see that the algorithm has determined a strategy for missing data. This is smarter ...
1
vote
0answers
67 views

Accounting for seasonality when using gradient boosting

I'm a novice attempting to predict automobile sales using a combination of previous sales (seasonal AR model), macroeconomic indicators such as CPI, consumer sentiment index etc. and more ...
2
votes
2answers
396 views

How to choose the number of trees in a generalized boosted regression model?

Is there a strategy for choosing the number of trees in a GBM? Specifically, the ntrees argument in R's ...
1
vote
0answers
44 views

How to approach prediction of new observations with incomplete data from model built from complete data

I currently have a gradient boosting model that uses the gbm package in R that classifies observations at the end of a year. Daily behaviors are logged for each ...
3
votes
1answer
82 views

How to communicate regression model performance to non-stats people?

In R I created a gradient boosted random forest from 100,000 records with 10 cross validation folds using the gbm library. I want to communicate the strength and ...
0
votes
1answer
238 views

Effect of features that are highly correlated with each other on a decision tree

I have a dataset of roughly 500 features and am training a binary classifier using GBM - gradient boosted machines, an ensemble of decision trees. Of these 500 variables, I am sure some are highly ...
0
votes
1answer
101 views

Negative value prediction using gradient bosting with Gaussian distribution in gbm

Why does gbm specified with Gaussian distribution throw negative values? Can it be controlled or modified if one does not want negative predictions (in ...
2
votes
1answer
95 views

What is the difference between the values in the 'fit' attribute of a gbm object and values computed by gbm.predict?

My intuition is that the fitted values and predicted values of a gbm object should be identical. But in this example with just one tree, the values are different: ...
4
votes
2answers
825 views

How does cross validation work in R's gbm package?

Can someone provide a work flow about this? For instance, suppose I am doing binary classification, For each iteration of the algorithm: Randomly sample k*N rows, where k is the bag.fraction, and N ...
1
vote
0answers
250 views

Machine Learning Algorithms vs. Linear Regression

Do machine learning algorithms like Boosted Regression Trees (in the R package (gbm)) follow the same statistical assumptions of not including correlated predictor variables in GLM? i.e. If I have ...
2
votes
1answer
502 views

has anyone implemented an autoencoder with random forests

I'm interested in exploring autoencoders which can be used to develop a compressed representation of data useful for machine learning. In my experience random forests are easier to work with and more ...
0
votes
0answers
81 views

Model Matrix and GBM

I am using the GBM package to create a prediction model. Can GBM handle categorical predictors, or do I need to create a model matrix before I input ?
4
votes
3answers
2k views

How to use R gbm with distribution = “adaboost”?

Documentation states that R gbm with distribution = "adaboost" can be used for 0-1 classification problem. Consider the following code fragment: ...
2
votes
1answer
684 views

GBM relative influence

For a regression problem, I used gradient boosting machines and assessed RMSE. My dataset is comprised of 34 features and 10,000 records. Only 2 predictors were considered 'important' (importance for ...
4
votes
1answer
830 views

Strategy to set the GBM parameters

I would like to know how you would set the parameters for the GBM model? 1) I could optimize the parameters sequentially. At first, using a large value for shrinkage and a small number of iterations ...
7
votes
2answers
897 views

Classification with GBM in R and imbalanced class sizes

I'm dealing with a supervised binary classification issue. I'd like to use the GBM package to classify individuals as uninfected/infected. I have 15 times more uninfected than infected individuals. I ...
1
vote
1answer
290 views

Does the 'fit' attribute of a gbm object contain the OOB estimates?

I have looked at the help (?gbm) and documentation for gbm.object, it only says that fit: a vector containing the fitted values on the scale of regression function (e.g. log-odds ...
2
votes
1answer
593 views

Role of n.minobsinnode parameter of GBM in R [closed]

I wanted to know what the n.minobsinnode parameter means in the GBM package. I read the manual, but it is not clear what it does. Should that number be small or large to improve the results?
4
votes
1answer
3k views

What are some useful guidelines for GBM parameters?

What are some useful guidelines for testing parameters (i.e. interaction depth, minchild, sample rate, etc.) using GBM? Let's say I have 70-100 features, a population of 200,000 and I intend to test ...
2
votes
0answers
320 views

How to calculate fitted values when using AdaBoost via gbm package in R? What does fitted values mean?

I poseted this question about two weeks ago. But it was closed because I cross-posted here and on SO. So I deleted the post on SO and re-post here. My question is as below. First of all I'm not ...
1
vote
0answers
446 views

How to calculate fitted values when using AdaBoost via gbm package in R? [closed]

First of all I'm not good at English. So please be patient to interpret my words. I really need your help. I used the R package GBM as probably my predictive modeling. I made a model with adaboost ...