Gradient Boosting Machine

learn more… | top users | synonyms

0
votes
0answers
32 views

Interpretation of partial dependence plots for multinomial GBM

I've been a big fan of the gbm package for some time, but am having difficulty understanding the output from the partial dependence plots in the case for multinomial classification problems. Below ...
1
vote
0answers
25 views

grouping attributes in RF and GBM

i have a dataset with 1000 samples and ~11k features (SNP markers). i have identified 100 additional binary features describing the markers themselves so i have a ...
0
votes
0answers
32 views

What happens to multi-category variables in algorithms like Random Forest that sample the feature space?

Suppose I have a multi-level categorical variable like color (say, with 7 levels). Some software libraries only allow numeric matrices to train models, so we need ...
0
votes
0answers
15 views

Metrics to evaluate the difference between gbm implementation

I am only interested in the case of a binary model. I count three different implementation of the gradient boost model: gbm from the ...
0
votes
0answers
18 views

Predict count data for unsurveyed areas

I am looking to predict count data from deer surveys for the unsurveyed areas. I want to make these predictions based on vegetation type and size of the vegetation type (acres). I started by using ...
3
votes
1answer
48 views

Interpretation of a gradient boost model

I recently did a gradient boost model to predict an event Y/N. I have a lot of features and a huge dataset. After a grid search cross validation, I manage to get an efficient enough model. (It is the ...
0
votes
0answers
42 views

Parallelizing GBM in R

I'm able to parallelize randomForest based on the foreach function in the following way: ...
1
vote
1answer
43 views

rebuild the tree generated by GBM package in R manually

I've used the gbm in R to generate a model. Although I can use predict.gbm to fit the model on new data set, I want to know the detailed step of gbm to calculate the prediction, beacuse I need to ...
0
votes
1answer
55 views

How is gbm package different from caret with gbm method?

I have a gbm problem and I am using the gbm package in R for it. But in most forums I see people using caret package for gbm. Is there any advantage of using caret instead of gbm package? If so, what ...
1
vote
0answers
29 views

gbm.fit() in R language

I have a data with size of 1200 rows having binary dependent variable and around 20 independent variables which are categorical as well as continuous in nature. I have tried machine learning technique ...
1
vote
1answer
42 views

Equal sampling for machine learning

I have a data with size of 1200 rows having binary dependant variable and around 20 independant variables which are categorical as well as continous in nature. I have tried 2 machine learning ...
1
vote
1answer
31 views

How to handle non- linear predictors in Decision Trees

I have a data set of just 1800 rows. I am not sure about how to provide inputs for certain variables. For eg., I have a column called number of previous jobs, and the values lie between 1 and 10, with ...
1
vote
2answers
168 views

Why the error on a training set is decreasing, while the error on the validation set is increasing?

When training XGboost model I observe the following outputs: ...
0
votes
1answer
36 views

Does the prediction/Scoring of model depends on the sequence of the variable?

Let's assume my dataset D1 has the variables x1, x2, x3, x4, x5, x6, x7, x8, x9, x10 I have used Gradient boosting regression on this and want to score on a new ...
0
votes
0answers
38 views

Calculating odds ratios when using gradient boosting approach

I always report odds ratios when using logistic regression for predictions. I wanted know is it meaningful to report odds ratios when modeling with gradient boosting approach? I am using gbm package ...
0
votes
0answers
47 views

Use lagged variable in tree-based model (GBM or RF)

I use GBM (gradient boosting model, R package: GBM) and RF(randomforest, R package: RandomForest) to predict a 0/1 binary variable (current disease status). My attributes (predictors) includes last ...
1
vote
1answer
55 views

R gbm: Lower shrinkage gives worse results?

I've read here and on other sites that when using GBM in R lowering shrinkage gives better results. Yet in my case this is clearly not the case. 0.1 is better than 0.01 with same amount of trees. Even ...
1
vote
0answers
30 views

Tolerance in boosted regression trees

I was interested in knowing if anyone is using the custom made function of BRT by Elith et al. (2008) in Journal of Animal Ecology "A working guide to boosted regression trees" and knows what does ...
0
votes
0answers
52 views

GBM Performance on different sampling techniques

I am working on a healthcare data set for breast cancer patients. This data set is class imbalances and the distribution of positive and negative classes is 80%/20%. In order to deal with the class ...
1
vote
1answer
159 views

Why use dummy variables in GBM using CARET library in R

I have seen a few examples implemting the gbm algorithm on youtube using the titanic dataset. These examples have turned some factor variables into dummy/indicator variables when GBM can handle factor ...
1
vote
1answer
45 views

can random forest project/interpolate based on new values of X?

Sometimes I want a model to predict what would happen when presented with values of predictor variables that it has not seen before. For example, say, I have predictor variables (X) that go from 1 ...
0
votes
0answers
48 views

how does predicted median go above 95% prediction interval when using GBM with quantile loss function

I was checking out how to create prediction intervals with Gradient boosted regression trees using Scikit-learn. If you set the alpha at .95 or .05, you can get the 95% prediction interval around the ...
4
votes
1answer
435 views

Gradient Boosting Tree vs Random Forest

Gradient tree boosting as proposed by Friedman uses decision trees as base learners. I'm wondering if we should make the base decision tree as complex as possible (fully grown) or simpler? Is there ...
1
vote
0answers
44 views

Does it make sense to minimize AUC when using GBM with weights?

I am using gbm(R's caret packages - using train function) on a class imbalanced data set with weights. So, class-1 has a weight of 1 and class-0 has a weight of 10. I am using parameter tuning and ...
1
vote
0answers
66 views

Meaning of Surrogate Split

I tried to figure but couldn't on what happens when missing values are present in some predictor variable and we have to solve the problem of regression using Random Forest. What is the meaning of ...
2
votes
1answer
84 views

Sampling : Gradient Boosting Tree

I have a question regarding the algorithm of Gradient Boosting Tree. I understand Simple tree is built for only a randomly selected sub sample of the full data set (random without replacement). Each ...
0
votes
0answers
59 views

error family in boosted regression tree: gbm package

I am trying to understand boosted regression tree. I am using the gbm package in R. I noticed that in this package one has to specify the error family. I understand ...
2
votes
0answers
56 views

How are base learners ensembled in a gradient boosting machine

I'm trying to understand how gbm's work at their core. So far, I believe that they run a basic decision or regression tree in the first step. Then they would work off of the residuals in the first ...
0
votes
2answers
31 views

Are Decision Function and Separating Hyperplane the same?

In many machine learning algorithms such as SVM, GBM, Logistic Regression, etc., are Decision Function and Separating Hyperplane the same?
4
votes
1answer
448 views

Intuitive explanations of differences between Gradient Boosting Trees (GBM) & Adaboost

I'm trying to understand the differences between GBM & Adaboost. These are what I've understood so far: There are both boosting algorithms, which learns from previous model's errors and ...
2
votes
0answers
59 views

Regression Tree Predictions

For various regression tree algorithms (e.g. GBM, Random Forest, Extra Trees), is there any sensible way to get predictions for new data when the independent variables for the new cases are much ...
0
votes
1answer
93 views

Repeating the same Random Forests through gbm is inconsistent?

I've noticed that when running a piece multiple times, the gbm produced (see below) produces slightly different results when viewing the summary. Should that be expected? I.e. running gbm regression ...
1
vote
0answers
119 views

Using gbm to eliminate variables before glm

I have a classification problem I am attempting to model using logistic regression (via the glm package in R): ...
2
votes
1answer
114 views

Choosing a sample rate for GBM models

I've created several GBM models to tune the parameters (trees, shrinkage and depth) to my data and the model performs well on the out-of-time sample. The data is credit card transactions (running into ...
1
vote
0answers
78 views

Formula to derive probabilities when using 'gbm' method through 'caret' package in R

I am training a classification problem by 'gbm' algorithm through 'caret' package in r.The response variable is a yes/nope type. Here 'objmodel' is the model I trained through method='gbm' and package ...
0
votes
0answers
365 views

Caret - Changing tuneGrid gives different results for same parameters

I'm having an issue training a GBM model in caret. I'm quite new to all of this, but I'll try to explain things as best as possible, but please let me know if you need any further info. My code looks ...
1
vote
1answer
1k views

Reconciling boosted regression trees (BRT), generalized boosted models (GBM), and gradient boosting machine (GBM)

Questions: What is the difference(s) between boosted regression trees (BRT) and generalized boosted models (GBM)? Can they be used interchangeably? Is one a specific form of the other? Why did ...
1
vote
0answers
118 views

Plot the training and cross-validation deviance [closed]

I'm running the gbm model using R caret package. To tune the model I used 10-fold cross-validation. I tried to get the following plots to guide my model tuning but didn't succeed: Plot the training ...
0
votes
1answer
844 views

R gbm package variable influence

I'm using the excellent gbm package in R to do multinomial classification, and my question is about feature selection. After deciding the number of iterations ...
1
vote
1answer
687 views

Change settings in the prediction model (caret package)

I am using the package caret and GBM method for my predictions. ...
1
vote
2answers
274 views

GBM: Predict the response variable measured in {0,20}

I need to predict the response that has values in {0,20}. Should it be used as a factor or as a numeric value? How does it influence on the prediction error? I am using GBM with the Gaussian ...
0
votes
0answers
17 views

Which distribution should I better use to predict the response in {0,20} applying GBM? [duplicate]

I want to predict the response that is in {0,20}. I am using GBM to make the prediction. ...
1
vote
2answers
339 views

Boosted trees and Variable Interactions

How can one see in a Boosted trees classification model, which variables interact with each other and how much? I would like to make use o this in R gbm package if possible
0
votes
0answers
75 views

GBM Prediction Interval Issue

I need to get prediction interval for GBM model (loss='ls'). I'm using this example as a basis http://scikit-learn.org/stable/auto_examples/ensemble/plot_gradient_boosting_quantile.html My model ...
1
vote
1answer
410 views

Training AUC and CV AUC in Boosted Regression Tree

My question is regarding the differences in the training data AUC score and the cross validation AUC score in boosted regression trees (BRT) built using the gbm.step function in the dismo package. I ...
0
votes
1answer
118 views

Different minimum observations per node in GBM model does not affect the AUC. How to explain?

My dataset is 6.3 million observations, with 150 features for each one. 25 000 of these observations are positive case and the rest is negative case, so about 1:250 class balance. I've been training ...
3
votes
0answers
333 views

Using an RMSE with derived confidence interval, to generate a prediction interval for an estimate

Previous questions have asked about creating prediction intervals for estimates derived from random forests or boosted regression trees, in a similar way to is easily achieved with linear regression ...
1
vote
0answers
156 views

How can I compare GBM feature importances to GBM partial dependence plots?

I am having trouble reconciling the difference between the indicated "importance" from a GBM that I am calculating with what is shown in the partial dependence plots. I would expect higher ...
1
vote
3answers
1k views

R: partial dependency plots from GBM package. Values and y-axis

I'm using the gbm.step package in R to look at the influence of three continuous variables on my continuous response variable. I have 234 observations. The model: ...
1
vote
1answer
763 views

Generating PMML export of a gbm model in R?

Is it possible to generate PMML of a gbm model? When I try to use the pmml library, I get an error: Error in UseMethod("pmml") : no applicable method for 'pmml' applied to an object of class ...