Gradient Boosting Machine

learn more… | top users | synonyms

0
votes
0answers
11 views

Difference in memory usage between gbm and blackboost [migrated]

I'm working on a database with around 250000 observation and 50 predictors (some are factors so in the end around 100 features) and I have trouble using the blackboost() function (from mboost package) ...
0
votes
0answers
18 views

modeling rates with machine learning tools (svm, gbm, nnet)

I have a numeric integer variable that is knowly proportional to an exposure measure plus other continuous / categorical covariates. If I were to use classical log-linear glms i would model ...
2
votes
0answers
63 views

Binary Class Distribution Effects on Probability Scores - (gbm) Boosted Tree Regression Models

Any help would be greatly appreciated. Problem: I need help to better understand the probability scores that come from the result of a decision tree model. Specifically, I'm using the gbm package ...
2
votes
0answers
61 views

Could you explain how gradient boosting algorithm works?

I have read a lot about gbm in Greedy function Approximation: A Gradient boosting Machine (pdf), but I can't code the algorithm for example LS_Boost in a simple way. Can someone explain what $h(x;a)$ ...
0
votes
0answers
16 views

Discretization of the split in random forest estimation

If the dataset I'm working with has too many independent variable dimensions, I may need to choose a proper subset of all possible locations in my feature vector to determine whether I am going to use ...
1
vote
0answers
41 views

Heuristic Feature Selection for Gradient Boosting

I originally posted on stackoverflow and was told to move it here: If I am trying to select from two different sets of features for a Gradient Boosting Machine but I do not want to run through ...
0
votes
0answers
24 views

Representing a class by negative number for training in GBM in R

I have already generated a huge train file of ~1GB in which the ground truth is one of 4 classes {-1,0,1,2}. Yes, I agree that I could have done it using 3 instead of -1, but for some conventional ...
0
votes
0answers
91 views

Measure the goodness-of-fit in boosted regression tree

What is the apropriate statistic to measure the goodness-of-fit in Boosted Regression Tree (or Gradient Boosting Regression) with continuous response? How can I calculate the coefficient of ...
1
vote
0answers
113 views

Significance of R Squared in Random Forest / GBM and GBM Tuning Parameters

I often get different level of responses when I discuss about R-Squared and its relevance to measuring the performance of a Random Forest or GBM model. In general, RMSE is a better and more ...
0
votes
1answer
125 views

Gradient boosting in R uses only a single variable

I am trying to build a boosting model using the package gbm in R. I have the following code: ...
1
vote
0answers
80 views

Mean Reciprocal Rank with GBM in R

Let's say I'm optimizing MRR with a GBM in R: ...
2
votes
1answer
243 views

Partial Dependency plots and Gradient boosting (GBM package)

Is it possible to plot a partial dependency plot to display the class probability and estimate the effects of a predictor for a GBM model? Something similar to ...
1
vote
0answers
60 views

How does gbm determine the missing node if the training set is complete?

I have a training set which is complete - no missing values, but yet if I pretty print one of the trees in gbm I see that the algorithm has determined a strategy for missing data. This is smarter ...
1
vote
0answers
56 views

Accounting for seasonality when using gradient boosting

I'm a novice attempting to predict automobile sales using a combination of previous sales (seasonal AR model), macroeconomic indicators such as CPI, consumer sentiment index etc. and more ...
2
votes
2answers
152 views

How to choose the number of trees in a generalized boosted regression model?

Is there a strategy for choosing the number of trees in a GBM? Specifically, the ntrees argument in R's ...
1
vote
0answers
34 views

How to approach prediction of new observations with incomplete data from model built from complete data

I currently have a gradient boosting model that uses the gbm package in R that classifies observations at the end of a year. Daily behaviors are logged for each ...
3
votes
1answer
73 views

How to communicate regression model performance to non-stats people?

In R I created a gradient boosted random forest from 100,000 records with 10 cross validation folds using the gbm library. I want to communicate the strength and ...
0
votes
1answer
165 views

Effect of features that are highly correlated with each other on a decision tree

I have a dataset of roughly 500 features and am training a binary classifier using GBM - gradient boosted machines, an ensemble of decision trees. Of these 500 variables, I am sure some are highly ...
0
votes
0answers
66 views

Negative value prediction using gradient bosting with Gaussian distribution in gbm

Why does gbm specified with Gaussian distribution throw negative values? Can it be controlled or modified if one does not want negative predictions (in ...
2
votes
1answer
69 views

What is the difference between the values in the 'fit' attribute of a gbm object and values computed by gbm.predict?

My intuition is that the fitted values and predicted values of a gbm object should be identical. But in this example with just one tree, the values are different: ...
4
votes
2answers
485 views

How does cross validation work in R's gbm package?

Can someone provide a work flow about this? For instance, suppose I am doing binary classification, For each iteration of the algorithm: Randomly sample k*N rows, where k is the bag.fraction, and N ...
1
vote
0answers
234 views

Machine Learning Algorithms vs. Linear Regression

Do machine learning algorithms like Boosted Regression Trees (in the R package (gbm)) follow the same statistical assumptions of not including correlated predictor variables in GLM? i.e. If I have ...
1
vote
1answer
385 views

has anyone implemented an autoencoder with random forests

I'm interested in exploring autoencoders which can be used to develop a compressed representation of data useful for machine learning. In my experience random forests are easier to work with and more ...
0
votes
0answers
62 views

Model Matrix and GBM

I am using the GBM package to create a prediction model. Can GBM handle categorical predictors, or do I need to create a model matrix before I input ?
4
votes
1answer
1k views

How to use R gbm with distribution = “adaboost”?

Documentation states that R gbm with distribution = "adaboost" can be used for 0-1 classification problem. Consider the following code fragment: ...
2
votes
1answer
462 views

GBM relative influence

For a regression problem, I used gradient boosting machines and assessed RMSE. My dataset is comprised of 34 features and 10,000 records. Only 2 predictors were considered 'important' (importance for ...
3
votes
1answer
540 views

Strategy to set the GBM parameters

I would like to know how you would set the parameters for the GBM model? 1) I could optimize the parameters sequentially. At first, using a large value for shrinkage and a small number of iterations ...
6
votes
2answers
582 views

Classification with GBM in R and imbalanced class sizes

I'm dealing with a supervised binary classification issue. I'd like to use the GBM package to classify individuals as uninfected/infected. I have 15 times more uninfected than infected individuals. I ...
1
vote
1answer
231 views

Does the 'fit' attribute of a gbm object contain the OOB estimates?

I have looked at the help (?gbm) and documentation for gbm.object, it only says that fit: a vector containing the fitted values on the scale of regression function (e.g. log-odds ...
2
votes
1answer
401 views

Role of n.minobsinnode parameter of GBM in R [closed]

I wanted to know what the n.minobsinnode parameter means in the GBM package. I read the manual, but it is not clear what it does. Should that number be small or large to improve the results?
4
votes
1answer
2k views

What are some useful guidelines for GBM parameters?

What are some useful guidelines for testing parameters (i.e. interaction depth, minchild, sample rate, etc.) using GBM? Let's say I have 70-100 features, a population of 200,000 and I intend to test ...
2
votes
0answers
273 views

How to calculate fitted values when using AdaBoost via gbm package in R? What does fitted values mean?

I poseted this question about two weeks ago. But it was closed because I cross-posted here and on SO. So I deleted the post on SO and re-post here. My question is as below. First of all I'm not ...
1
vote
0answers
383 views

How to calculate fitted values when using AdaBoost via gbm package in R? [closed]

First of all I'm not good at English. So please be patient to interpret my words. I really need your help. I used the R package GBM as probably my predictive modeling. I made a model with adaboost ...
8
votes
2answers
2k views

What does interaction depth mean in GBM?

I had a question on the interaction depth parameter in gbm in R. This may be a noob question, for which I apologize, but how does the parameter, which I believe denotes the number of terminal nodes in ...