Tagged Questions

Gradient Boosting Machine

learn more… | top users | synonyms

1
vote
0answers
23 views

Using an RMSE with derived confidence interval, to generate a prediction interval for an estimate

Previous questions have asked about creating prediction intervals for estimates derived from random forests or boosted regression trees, in a similar way to is easily achieved with linear regression ...
0
votes
0answers
10 views

Combining collinear continuous predictor variables in GBM

I'm dealing with a dataset (n=254) with one dependent variable (Y), and three independent variables (X1, X2, X3), all continuous. I would like to compare the contribution from each IV to Y. I've been ...
0
votes
0answers
16 views

How can I compare GBM feature importances to GBM partial dependence plots?

I am having trouble reconciling the difference between the indicated "importance" from a GBM that I am calculating with what is shown in the partial dependence plots. I would expect higher ...
0
votes
0answers
13 views

R: selecting appropriate fit statistics from GBM output

I'm using the gbm.step function in the dismo package in R to evaluate the contribution of three continuous variables on my ...
1
vote
1answer
45 views

R: partial dependency plots from GBM package. Values and y-axis

I'm using the gbm.step package in R to look at the influence of three continuous variables on my continuous response variable. I have 234 observations. The model: ...
0
votes
0answers
8 views

Y axes on the logit scale and centered in gbm.plot [migrated]

I am currently exploring the gbm functions in the package dismo to create boosted regression trees for species distribution modeling. I have been using the dismo vignettes as well as the 2008 paper "A ...
-1
votes
0answers
36 views

Queries in GBM Model in R

I have the following questions pertaining to GBM Model in R. Question 1 : I have a mixture of factorial and numerical data. And Its a binary classification ...
0
votes
0answers
25 views

Parameter selection for GBM

I'm building a Gradient Boosting model. Given a dataset and event rate, is it possible to get a formula/ definitive strategy for the optimum number of trees, shrinkage parameter and depth of trees? I ...
0
votes
1answer
113 views

Generating PMML export of a gbm model in R?

Is it possible to generate PMML of a gbm model? When I try to use the pmml library, I get an error: Error in UseMethod("pmml") : no applicable method for 'pmml' applied to an object of class ...
1
vote
3answers
187 views

Performance drop between training and validation datasets

I have been using R's GBM (Gradient Boosting Machine) package for several months. I typically split my data into three partitions: Training, Validation, and Testing. I use the validation data set to ...
0
votes
0answers
74 views

Anyone use R or Python as statistical tool in industry? How to get coefficients of gradient booting models?

I tried gradient boosting models using both gbm in R and sklearn in Python. However, neither of them can provide the coefficients of the model. For gbm in R, it seems one can get the tree structure, ...
1
vote
2answers
50 views

Estimating expected lifetime from hazard ratio and estimated base hazard function

Apologies if this is a basic question, I am not very familiar with survival analysis ... I have trained a gradient boosted Cox proportional hazards model in R, and have been able to obtain reasonable ...
0
votes
1answer
69 views

GBM Bootstrap Prediction Interval Code Error

based on code presented in thread: How to find a GBM Prediction Interval I am trying to apply this to my dataset. Below is my full code, and I am having issues with the bootstrap function. ...
2
votes
1answer
123 views

How to find a GBM Prediction Interval

I am working with GBM models using the caret package and looking to find a method to solve the prediction intervals for my predicted data. I have searched extensively but only come up with a few ...
1
vote
0answers
38 views

Is multicollinearity a problem with gradient boosted trees (i.e. GBM)?

A question about multicollinearity for random forests has been asked and answered, but what about boosted trees?
0
votes
0answers
48 views

Pitfalls of using random forest/GBM on proportion data?

I have a set of data with a dependent variable which represents a proportion, and many of the samples contain a response of 1. I would like to build a random forest or GBM regression model on the ...
0
votes
1answer
16 views

Can adaboost choose the same variable for multiple splits for a given tree?

Can adaboost choose the same variable for multiple splits for a given tree? The model was given 100 + variable to choose from and it did choose them for the other trees in the ensemble. I am using ...
1
vote
0answers
45 views

How does GBM model handle categorical variables with many levels

I am using gbm model to fit a continuous dependent variable Y with several categorical variables, say, X, Z, V, and W. Suppose X has many levels (distinct values) and Y has moderate number of levels, ...
0
votes
0answers
23 views

GBM package: Why there is a missing node?

Why there is a missingNode as 3 as there are no missing values? I have the data in the following form: ...
0
votes
0answers
79 views

How to interpret the output of a multinomial classification model in R package gbm

After running a gradient boosted model with n data points using multinomial regression where the response variable (a factor, as required by the gbm function) has ...
0
votes
0answers
15 views

Finding the effects of certain levels of a factor predictor

I have fitted a binary classification gbm model, and one of the predictor variables, Affiliate has 50 different levels. Given ...
0
votes
0answers
45 views

Sample Weights for classification using Gradient-Boosted trees?

How can "weights" be given to different samples according to their relative importance while using Gradient boosted decision trees for classification? How does the ...
0
votes
0answers
131 views

gbm R multinomial vs bernoulli

I am using the gbm package to fit a binary variable using several attributes, some numeric and some categorical. Since the output varible was defined as factor I initially did ...
1
vote
1answer
59 views

gbm.perf with method = “test” returns n.trees from last run

As the title says, I'm getting some interesting results from gbm.perf. The first time I ran into trouble was after a run where n.trees was set to 7,000. When gbm.perf also returned 7,000 I got ...
0
votes
0answers
29 views

Estimating the running time of gbm grid tuning

I am trying to estimate the time it will take me to run a tuning grid on gbm (I am using the R Caret package but this is irrelevant as I am interested in the relative processing time). I can see many ...
0
votes
0answers
62 views

BRT analysis using count data

I have some problems with my BRT analysis. Introduction to the data: The dependent variable is count data of a specific palm species in SA, and the predictors consists of nine various kinds of ...
3
votes
2answers
188 views

How to find optimal values for the tuning parameters in boosting trees ?

I realise that there are 3 tuning parameters in the boosting trees model, i.e. the number of trees (number of iterations) shrinkage parameter number of splits (size of each constituent trees) My ...
1
vote
0answers
63 views

How do i estimate the Weights of the predictions assigned to each of the tree in GBM using R? How does GBM split nodes?

I ran a GBM model in R with loss function as bernoulli and n.trees=1000. I want to see the weights assigned to the predictions coming from 1000 trees. Is there any command in R that does that? How ...
1
vote
1answer
104 views

Negative predictions for binomial predictions from gbm in R

I've just fit a binomial model (training y = 0 or 1) using R's gbm package. When I calculated predicted values using my validation data, some of the predicted values were less than 0. Is this normal ...
1
vote
2answers
118 views

Combine decision trees from GBM to reduce output

I am curious if any research has been conducted to efficiently combine trees resulting from a gradient boosting process. I routinely run a process that generates 20 or 30 thousand trees in R. I then ...
1
vote
0answers
56 views

R: Which distribution to use with gbm for gamma distributed data?

When I use GLMs I can use the option family="Gamma" for analysing data consisting of positive real numbers. Also package gbm ...
5
votes
3answers
426 views

Why doesn't Random Forest handle missing values in predictors?

What are theoretical reasons to not handle missing values? Gradient boosting machines, regression trees handle missing values. Why doesn't Random Forest do that?
2
votes
1answer
142 views

How can I modify default parameters of a gbm.step plot?

I am using the function gbm.step() from the dismo package to assess the optimal number of boosting trees using k-fold cross ...
1
vote
0answers
50 views

Random Forest and Factor Predictors [duplicate]

How do decision tree based ensembles like random forest deal with categorical ("factor") predictor variables? My guess would be that indicator variables are created for each factor via a ...
0
votes
0answers
80 views

modeling rates with machine learning tools (svm, gbm, nnet)

I have a numeric integer variable that is knowly proportional to an exposure measure plus other continuous / categorical covariates. If I were to use classical log-linear glms i would model ...
2
votes
0answers
156 views

Binary Class Distribution Effects on Probability Scores - (gbm) Boosted Tree Regression Models

Any help would be greatly appreciated. Problem: I need help to better understand the probability scores that come from the result of a decision tree model. Specifically, I'm using the gbm package ...
2
votes
0answers
154 views

Could you explain how gradient boosting algorithm works?

I have read a lot about gbm in Greedy function Approximation: A Gradient boosting Machine (pdf), but I can't code the algorithm for example LS_Boost in a simple way. Can someone explain what $h(x;a)$ ...
0
votes
0answers
31 views

Discretization of the split in random forest estimation

If the dataset I'm working with has too many independent variable dimensions, I may need to choose a proper subset of all possible locations in my feature vector to determine whether I am going to use ...
2
votes
2answers
175 views

Heuristic Feature Selection for Gradient Boosting

I originally posted on Stack Overflow and was told to move it here. If I am trying to select from two different sets of features for a Gradient Boosting Machine, but I do not want to run through ...
0
votes
0answers
41 views

Representing a class by negative number for training in GBM in R

I have already generated a huge train file of ~1GB in which the ground truth is one of 4 classes {-1,0,1,2}. Yes, I agree that I could have done it using 3 instead of -1, but for some conventional ...
0
votes
0answers
198 views

Measure the goodness-of-fit in boosted regression tree

What is the apropriate statistic to measure the goodness-of-fit in Boosted Regression Tree (or Gradient Boosting Regression) with continuous response? How can I calculate the coefficient of ...
1
vote
0answers
220 views

Significance of R Squared in Random Forest / GBM and GBM Tuning Parameters

I often get different level of responses when I discuss about R-Squared and its relevance to measuring the performance of a Random Forest or GBM model. In general, RMSE is a better and more ...
0
votes
1answer
184 views

Gradient boosting in R uses only a single variable

I am trying to build a boosting model using the package gbm in R. I have the following code: ...
1
vote
0answers
175 views

Mean Reciprocal Rank with GBM in R

Let's say I'm optimizing MRR with a GBM in R: ...
2
votes
1answer
661 views

Partial Dependency plots and Gradient boosting (GBM package)

Is it possible to plot a partial dependency plot to display the class probability and estimate the effects of a predictor for a GBM model? Something similar to ...
1
vote
0answers
123 views

How does gbm determine the missing node if the training set is complete?

I have a training set which is complete - no missing values, but yet if I pretty print one of the trees in gbm I see that the algorithm has determined a strategy for missing data. This is smarter ...
1
vote
0answers
69 views

Accounting for seasonality when using gradient boosting

I'm a novice attempting to predict automobile sales using a combination of previous sales (seasonal AR model), macroeconomic indicators such as CPI, consumer sentiment index etc. and more ...
3
votes
2answers
460 views

How to choose the number of trees in a generalized boosted regression model?

Is there a strategy for choosing the number of trees in a GBM? Specifically, the ntrees argument in R's ...
1
vote
0answers
49 views

How to approach prediction of new observations with incomplete data from model built from complete data

I currently have a gradient boosting model that uses the gbm package in R that classifies observations at the end of a year. Daily behaviors are logged for each ...
3
votes
1answer
90 views

How to communicate regression model performance to non-stats people?

In R I created a gradient boosted random forest from 100,000 records with 10 cross validation folds using the gbm library. I want to communicate the strength and ...